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ABSTRACT 


A current trend in evaluation to balance quantitative 
measures and qualitative judgments provided the philosophical context 
for this case study. In the late 1960's a balanced methodology was 
devised to evaluate doctoral programs in the School of Education at 
Indiana University. The model of doctoral education that emerged 


facilitated analysis, but it was contested by many faculty members. 
Students constituted the input and output variables in the model. The 
Major process variables were stated as clusters of student activities 
while university resources constituted context variables. A statement 
of formal external criteria rather than substantive criteria was 
formulated that could be applied to all doctoral programs. 
Operational indicators of the criteria were enumerated by a faculty 
committee. The initiation of formal program review utilizing the 
evaluation process guide was postponed for six years and is still in 
halting progress. The process of developing the guide, the guide 
itself, and the problems of impiementation at Indiana University 
provide a history of a difficult undertaking which proved 
particularly susceptible to the effects of circumstance. (KB) 
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Curriculum Elements of Graduate Education: 
A Brief Case Study on the Evaluation of a Doctoral Program 


by Thomas M. Schwen 


This paper contains a brief, interpretive, case study focused on 
the methodology used to evaluate a doctoral program in educational 


technology. In the recent literature on evaluation the distinction 


between decision making and judgment is often made (e.g., Scriven [2]): 


Until recently we, as educators, have written and behaved as if our 
scientific procedures permitted us to make yes/no, pass/fail, no/no go 
decisions. Our criterion referenced tests, attitude surveys, prototype 
testing, etc., lead to decision matrices that permitted the clear 
resolution of our problems in a network of dichotomous or, with 
sophistication, trichotomous decisions. Of course, this is an over- 
statement, but it is hard to deny the influence of rampant empiricism 
that constitutes our common past. 

Our colleagues in evaluation have helped to free us from the 
narrow minded empiricism that has restricted our thinking. It is 
becoming, incveasingly, legitiwaee to make judgments about the important 
phenomena in our professional areas of concern. It is conceivable that 
an evaluation of a curriculum, instructional products, development pro- 
cesses and the like can include 'qualitative' data and that the 
considered judgments following such an evaluation will be considered 
‘significant' without reference to an analysis of variance table. 

If the reader will forgive the hyperbole, patently redundant 
hyperbole, at that, we seem to be entering a period to which balance 
is an important principle. A whole host of book announcements tantalize 
in a weekly barrage of announcements concerning: action oriented 
evaluations, evaluation dictionaires, conclusion oriented case studies, 


and the like. The forces of empiricism and judgment would seem to be 
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in a state of compromise. Appropriate matching of qualitative method- 
ologies to empirical problems is a phrase which seems to describe the 
current posture of evaluation efforts. 

This introduction is, of course, a caveat which places the 
following abbreviated case study in context. The Division of 
Instructional Systems Technology (hereafter DIST) is one of five 
administrative units in the School of Education (hereafter School) at 
Indiana University. It is, also, one of the largest doctoral programs 
in Education at Indiana and the largest in Educational Technology in 
the nation. Beginning late in the 1960's a series of events prompted 
the program leadership to engage in a lengthy and comprehensive self 


evaluation of its graduate curriculum. The formal evaluation activity 


began in ernest in 1972 and is, technically, still in progress. 


The methodology of the evaluation is the focus of the case rather 
than the results. As noted, the evaluation is still in progress. Also, 
for other reasons, which will become obvious, it is the author's judgment 
that there is more to be gained from a review of the methodology than 
from a sampling of the rather extensive results. 

The methodology grew out of conscious attempts to come to grips 
with the complex reality of the doctoral program in such a manner that 
would avoid under-representation of the abstract and complex forces 
that play on it, on the one hand, and obfuscation of weaknesses and 
strengths by persuasive or subjective instrumentation, on the other. 

All parties involved were in substantial agreement on the principle of 
balance between subject and objective data. The inevitable differences 
of opinion focused on the operationalization of the criteria and questions 
which followed. The nature of this conflict and the compromises which 


resulted seem to offer a profitable example for consideration. 
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Background 


Before the methodology is discussed some contextual information is 
in order. As noted above, the DIST incorporates one of the largest 
doctoral programs in the School of Education. Twenty three doctoral 
level faculty serve the program. These faculty are all on joint 
appointments with either the Audio-Visual Center or other academic 
units in the School. The School funds approximately 11 full-time- 
equivalent positions. Approximately 100 doctoral students are in 
residence at any one time and the median for degrees awarded has been, 
approximately, 20 per calendar year in the last five years. The 
students have been recruited from and placed in all fifty states and 
over 100 foreign countries. Approximately 20% (20 per year) of the 
students have been from foreign countries in rec.at years. 

The students may elect to concentrate in or combine aspects of 
four emphasis areas: Message Design (production), Instructional 
Development, Diffusion and Adoption of Educational Innovations, and 
Administration of Learning Resources Programs. Also, the student 
may combine the above with other thematic specializations within the 
DIST such as research. Finally, other studies outside of the Division 
such as Evaluation, Educational Psychology, Radio and Television, 
Organizational Behavior (Business), etc., are required in the doctoral 
program. In point of fact, there are probably no -two doctoral programs 
that have included the same profile of studies. Although the staff of 
the Division almost universally considers this state of affairs an 
advantage, there are disadvantages in managing and articulating the 
program to students and evaluators, for that matter. 


The content categories named above are reduced to four functional 
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categories in planning individual courses of study. Each student is 
required to complete: an 18 semester-hour-core of inquiry skills 

(e.g., statistics) and-analytic perspectives (e.g., philosophy); a 45 
semester hour major (a combination of the emphasis areas) and two 15 
hour minors (e.g., Evaluation, Radio and Television, etc.). The ideal 
chronology for an individual program would include one post-bachelors 
year of introductory coursework, a second year of advanced seminars 

and skill building courses, and a third year of qualifying exams, 
dissertation seminars, research, and completion of the degree. The 
three year pattern does not describe the majority of students in residence. 
It is quite common for students, to spend all or portions of a fourth 
year completing their work. Most students are employed in funded 
internships by either the Audio-Visual Center or other development 
laboratories during their academic tenure at Indiana. These internships 
account for much of the deviation from the three year pattern. Another 
deviation from the pattern may be seen in the 50% of the students who 
transfer some portion of their graduate program to Indiana from other 


institutions. 


The DIST faculty developed the aforementioned curriculum pattern 


in the period from 1968-1971. Although many factors influenced that 
curricular revision it is possible to argue that there were two primary 
concepts that motivated the revision. (1) The concept of technology 
was undergoing rapid change. The concepts of process such as systems 
analysis, instructional development, message design, and diffusion and 
adoption were quite prominent in the new course offerings developed at 
that time. (2) Also,*the concept of the RDD&E model clearly influenced 
the curriculum planners. The older specializations of Utilization, 


Production and Management‘were replaced by: a Research theme; the 
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Instructional Development, Message Design, and Diffusion and Adoption, 
emphasis areas. The management category was retained and early attempts 
to organize an evaluation component of the curriculum were superceded 
by developments outside the divison. (As noted above this area is a 
minor chosen by some of our students.) At one time the faculty listed 
35 experimental courses during this process of revision. At.the end 

of the active period of curriculum revision 12 new courses had been 
added to the curriculum. Of the current 37 courses offered, (Appendix 
D) six of the old courses were thoroughly revised. Of the 19 remaining 


courses, it is fair to observe all were substantially modified. These 


necessarily sketchy details are a pale representation of the intellectual 


ferment that characterized that period. In any event, by the end of 
that period most faculty and students were quite anxious to assess the 
changes in the curriculun. 

In addition to the internal push for evaluation, a series of events 
outside the Division helped bring about the evaluation process. In 1969, 
the Graduate School initiated a review of all Graduate Programs offering 
the Ph.D. The first experiences with that review were quite controversial. 
Two programs in the School lost their Ph.D. status: as a result of the 
ensuing furor the review process was halted and the School subsequently 
initiated its own review. Essentially, the argument was that the 
School would conduct a more thorough review than the Graduate School 
and then return to the Graduate School reviews. As of this writing the 
School reviews have been embroiled in their own controversy, and with 
a change in administration, the process created by the School would 
appear to have been tabled. The controversy created by review processes 
did not deter the leadership of DIST. The faculty has completed the 


first phase of the School review and is awaiting a formal response from 
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the School administration. This review process will be explained in 
more detail below. Also, the significance of the controversy surrounding 


the review processes will be incorporated in subsequent discussions. 


Methodology 

It should be noted that a number of individuals and committees 
contributed to the methodology described below. Egon Guba, an early 
advocate of the CIPP model (3) was most influential in developing the 
conceptual structure for the evaluation. The structure was created by 
a statement of formal criteria that applied to the evaluation of all 
doctoral programs (Appendix A). Guba deliberately chose formal 
(external) rather than substantive (intrinsic) criteria. This decision 
allowed him to avoid the cost and political problems of bringing out- 
side experts to make substantive judgments about doctoral programs on 
the one hand, or to ask the School's faculty to make substantive 
judgments concerning programs outside their areas of expertise, on the 
other. In other words, he argued that there are a set of formal 
criteria (with accompanying data) that allow faculty with divergent 
backgrounds to make judgments about the strengths and weaknesses of 
doctoral programs. This was thé seminal argument of the evaluation 
process. Once the faculty agreed to this assertion, any other arguments 
could be resolved by a process of compromise. The nature of the formal 
criteria will be discussed in more detail in the material which follows. 

The Advanced Graduate Studies Committee (1969-70) of the School 
was responsible for suggesting means of operationalizing the criteria 
(Appendix B). The usual conditional statement concerning credit due 


the original authors and this author's responsibility for misinterpre- 


tation and misrepresentation is appropriate here. 
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Conceptual Structure 


Guba's (Table 1) formal criteria were, obviously, designed from 


the perspective a systems model such as CIPP (3). 


Insert Table 1 about here 
It is interesting to note, however, that this superordinate structure 
(figure 1) was never explained to the School's faculty at large. The 
structure represented in figure 1 is an interpretation by this author 


and his assistants.* 
Figure 1 about here 


The systematic concept of the doctoral program is, obviously, 
student oriented. Students are input and output variables with the 
major process variables stated in terms of clusters of student activities. 
The Context variables are a collective of more traditional administrative 
concerns. This structure is deceptively simple. Although it had a 
persuasive logic or integrity about it, the program faculties did 
express concern when using the structure to make judgments about their 
curricula. 

Some faculty were concerned about the basic assumption of the 
model. Students are certainly important even primary outputs or products 
of any curriculum or program. However, there are many other program 
outcomes that a different systematic concept could have elucidated 
such as, national leadership, influence on other curricula in the same 


or related disciplines, service to the School and University, etc. 


wee ew em w ewe mw em ee ee we we ew we ee we eee o-- 


* A special acknowledgment is given to Dr. Gene Wilkinson, Assistant 
Professor of Education at the University of iid who was formally 
an assistant to the author. 
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In other words, some faculty were concerned about underrepresentation 
of their strengths. 

There were other concerns as well. Casting the faculty, fiscal 
resources and competencies as Context rather than Process variables 
created some dissonance. The conflict is difficult to summarize, 
however, conceptualizing the variables in this fashion seems to cast 
them in a less dynamic or more static framework. To many faculty 
their budgets, general goals or program competencies, and their 
colleagues were at least as dynamic in their personal concepts of 


program as student development and inquiry activities. 


After the fact, these criticisms can be reconstructed in a more 


articulate fashion than they were originally stated. In the 
negotiation about the process each program was permitted to redesign 
its own model and procedures. None of the 21 doctoral programs made 
a modification of the conceptual structure. There is little doubt in 
the author's mind that much of the early dissonance can be associated 
with personal and collective anxiety on the part of the faculty. 

The structure had many strengths. Any social science analytic 
process destroys some aspect of the reality it examines (Schwen [1]). 
In this case, despite the aforementioned weaknesses, the model imposed 
a concept of the doctoral program which created a rigorous and rather 
complete student perspective. The structure captured the essential 
elements of the doctoral program in a fashion which permitted a.continuous 
dynamic concept of the student program to emerge. It was, for example, 
far easier to articulate the logical and empirical connections between 
inquiry training and the scholarly production of graduates or the 
link between practicum activities and the types of positions graduates 


occupy. In the case of the DIST it forced the faculty to create a more 
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articulate set of interconnections between all its diverse program 


elements. 


In summary, the criticisms of the conceptual structure of the 


model focused on its omissions. Its enduring strength after critical 
analysis was probably due to its simple but persuasive perspective 

on the 'studenting' process. Also, it is fair to acknowledge that a 
single articulate argument will often be more persuasive than anxious 


diffuse responses. 


Criteria 
The criteria listed in Table 2 are stated in the manner in which 


they were operationalized. The original statement of criteria (Appendix A) 
Insert Table 2 about here 


separated competency-based and multiple experience program attributes 
into two criteria (#3). The reader is advised to see Appendix A for 
a more complete discussion of all criteria. This writer will attempt 
to summarize the faculties' reactions, particularly the DIST faculty 
reaction. The first doctoral program to test the process was the 
English Education faculty. In retrospect, it is interesting to note 
that their response signaled much of the negative reaction that wouid 
follow. The initial reaction, one repeated by all groups that used 
the process, was that each of the criteria were more or less 
defendable but that the amount of time that it took to collect data 
on all criteria outweighed the advantages of the review. This group 
raised another issue which reverberated throughout the rest of the 
School. They, took singular exception to criterion two. The group 


interpreted the phrase competency-based as meaning behavioral objectives. 


11 


They argued in conjunction with the position of their national associa- 
tion, that behavioral objectives are overused and they would misrepresent 
the complex understandings that they were attempting to develop in their 
students. Many of the early exchanges were quite heated. Finally, 
the program modified its procedure of reporting to include goals or 
lists of understandings. This modification was of the type anticipated 
by the evaluation documents. The intensity of the reaction probably 
signaled the anxiety of the faculty as well as a genuine intellectual 
issue. 

There were diffuse reactions to the criteria by other faculty groups. 
Most faculty groups had not formally designed self renewal plans (#10) 
and so the review process was, in effect, asking them to modify their 
collective behaviors. There were many specific intellectual objections 
to that criterion that could probably be summarized by the observation 
that graduate faculty groups are quite independent of administrative 
authority and they are slow to modify their departmental procedures. 
The only other substantive reaction observed by this author was to 


criterion 3 (individualization). Again, there were a number of diffuse 


responses that, probably, marked a more basic objection. Most doctoral 


faculty's personal concept of the doctoral program is one of a series 
of personal interactions between themselves and their students. To 
many faculty this is the epitome of individualization, therefore, -the 
statement of this criterion seemed overly formal and redundant. 

The DIST reactions to the criteria were fairly typical. This 
was surprising, initially. The author had not anticipated that the 
faculty would object to the statement of a competency based curriculum. 
There seemed to be two parts to the argument. To many, the process 


was disfunctional because of its complexity. They argued that the long 
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lists of behavioral objectives would do more to obfuscate the curriculum 
pattern then to clarify it. (The reader is reminded that this 

dialogue took place in 1972-73 before the recent widespread disenchant- 
ment with behavioral objectives was in vogue.) In addition, some 
faculty took the English Education faculty's position; they agreed that 
many of the understandings taught in the curriculum could not be 
appropriately defined in behavioral terms. They argued that complex 
evaluation behaviors, for example, would be distorted or misrepresented 


in the behavioral objective format. These arguments had been latent 


or unspoken before the review. As noted above, they surprised the 


author, at the time, however, the dialogue ultimately resulted in a 

better understanding of the issues involved. The DIST faculty were 

somewhat more favorably disposed to Criterion 3 (Individualization) 

because of the size of the faculty and the diversity of curriculum 

experiences. They essentially felt that this was a strength of the 

program, but the complexity of the situation suggested careful analysis. 

Also, the faculty felt that criterion 10 (self renewal) would allow the 

program to explicate the recent curriculum review. Finally, criterion 

9 (resources) presented some technical problems, because the DIST 

budget was not clearly separated from the A-V Center budget at that time. 
In summary, neither School's faculty at large or the DIST faculty 

raised many substantive criticisms to the criteria. The objections 

that did arise, e.g., criterion 2, (competencies) were ameliorated 


by changes in the method of reporting. 


Implementation of the Review 


The concept of the review was clearly explicated as formative in 
nature. The program areas were to develop long term data collection 


procedures that would permit a replication of the procedures on a five 
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year basis. The administration proposed that the assessment would 

lead to negotiations where strengths and weaknesses would be reviewed 
and remedial action would be mutually determined where necessary. 
Although the formative concept was clearly stated, it would appear 

that it was not persuasive. Many individuals and groups of faculty 
were, obviously, defensive. The faculty were quite creative in delaying 


the process. There were many long meetings devoted to explaining the 


process and still more meetings clarifying misunderstandings. In 


addition, the English Education Review (the first trial of this process) 
lasted at least three times longer than had been planned. The faculty 
council reviewed evaluation documents at various times further delaying 
the-process. Of course, it'is difficult to separate anxiety from 
genuine intellectual responses. The author is willing to assert that 
six years of dialogue concerning the efficacy of the process may be 
‘interpreted as more than a purtipiownteT discussion of a commonly agreed 
upon goal. 

The mechanism that was proposed for implementing the evaluation 
involved five steps. (1) The program faculty were to be assembled and 
introduced to the evaluation process. (2) The faculty would review 
the process, make adaptations where necessary and organize its members 
to complete the process. (3) The implementation and write up would 
follow,running from three to six months. (4) The completed document 
would be delivered to a group of three faculty reviewers. (This review 
panel would have been selected by the program faculty undergoing the 
evaluation.) The reviewers would spend one to three months analyzing 
the report and summarizing their reactions in writing. They would be 
allowed to communicate with program and request additional information 


or revisions. (5) Finally, a three way meeting between the program 
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leadership, the review board chairperson, and the Associate Dean for 


Graduate Study would summarize the process and decide on what remedial 


actions were necessary, if any. The DIST has completed phases 1-4. 


Instrumentation 

The operationalization of the criteria has been summarized in 
Table 2. A more detailed discussion of the procedures may be found 
in Appendix B. As suggested earlier, there was a great deal of 
controversy concerning these procedures. There were both general and 
specific criticisms. The most general and wide spread response was 
that the procedures required: much more time than the faculty could 
reasonably invest or at least a dleproportionwee investment for the 
amount of return. The administrative response was that most information 
could be gathered by clerical staff and that once the information 
procedures were established, subsequent evaluations would proceed more 
efficiently. In addition, the administration argued that the evaluation 
process should enjoy a high priority, especially, in light of the 
experience with the Graduate School Reviews. Finally, the administration 
argued that specific modifications were always possible in accumulating 
the types of evidence. The administrators appeared to win the argument; 
the process continued, with more time allowed to complete each phase, 
especially phases 1, 2, and 3. The DIST faculty volunteered to be a 
part of the first complete cycle of evaluations. The evaluations were 
to be cycled on a five year basis, one fifth of the doctoral programs 
beginning the process in any one calendar year. 

The specific criticisms of the procedures for collecting data were 
quite numerous. The request for a common listing of competencies (#3) 


hierarchically ordered (#4, #13) created a great deal of dissonance. 
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The arguments were an extension of the general problem with the 
competdhey based criterion. In addition, the specific suggestions for 
validating the hierarchy (Appendix B) were questionable at best and 
probably invalid. Most. programs, including the DIST, developed a set 
of logically derived goals and competencies without attempting to 
validate the structure. 

The analyses of five years of students via their records and by 
questionnaire was, also, a subject of concern. For some programs, 


including the DIST, this amounted to 100 students or more. For the 


smaller programs this constituted a very small sample. In other words, 


the sample was too large in some cases and too small in others. 

Many programs had not systematically catalogued their 
curriculum experiences, e.g., #3, #11, #13, #15, etc., therefore, there 
was some general complaints about the inconvenience. 

Many criticisms, as noted previously, were a function of the 
resistance to initiating new means of documenting activities, e.g., 
#2, the self-evaluation plan; #9, analysis of selection criteria; etc., 
the data collection procedures were in effect calling for more complete 
documentation of doctoral programs than was common in previous rractices. 
There were many individual adaptations to the suggestions and there 
was general encouragement for this trend by the administration. They 
reasoned that the recommendations were just that and there would be 
many alternate methods for operationalizing such abstract criteria. 

The DIST collected most of the data suggested in Table 2. The 
major exception was with #4, the hierarchical statement of competencies. 
As noted, the faculty used a logically derived hierarchical statement 


of goals within each emphasis area. A core of competencies was also 
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developed; it included the general understandings and skills required 
of all students and the common research or inquiry skills required 

of all students. There were some embellishments, e.g., #18, evaluation 
of faculty; there were several surveys and related reports available 


ranking the major educational technology programs. The faculty incorporated 


several of these in its report. 


Organization of the Process 

The organization of the data collection procedures was the 
responsibility of this author. It was a rather complicated affair 
due to the extent of data required and the fact that much of the 
individual categories of data related to several criteria. After some 
deliberation a PERT (Appendix C) system was developed to coordinate 
the procedures. Also, individual faculty members were assigned to 
develop and write chapters of the report which coincided with the 10 
criteria. The PERT process was only minimally successful. The DIST 
faculty were generally supportive; however, there were many delays in 
the development of competency statements, the statement of evaluation 
procedures, and the statement of development and inquiry activities. 
These first order activities required a great deal of small group and 
individual effort. The inevitable delays associated with this process 
created further delays in the preparation of the questionnaire and 
other activities further down the critical PERT paths. This author 
and the DIST leadership seriously underestimated the time required to 
finish the project. The data collection phase took approximately 18 
months to complete. 

Although the positions of the School's faculty and administrators 


has been delineated in a logical fashion (More or less!), the dialogue 
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was not ordered or logical in many instances. As noted above, the 
evaluation process seemed to be a focus of considerable anxiety. 

| There were many misinterpretations of the documents, the communiques 
that were exchanged, etc. The DIST faculty remained reasonably aloof 
from this dissonant dialogue. Most faculty felt that they were dealing 
from a position of strength. The Division chairman and other leaders 
argued that the program was strong and whatever weaknesses were 
uncovered could be ameliorated. Many of the larger more established 
doctoral programs of the School took the same position. 

In delineating the 'reality' of the evaluation process it is 

difficult to overestimate the group dynamics that influenced the progress 


of the evaluation. The process was initiated during a period of stress 


(two doctoral programs losing Ph.D. status). Many faculty groups 


seemed to perceive the process as a frontal attack on their collective 
future. In addition, the process was quite complex and cumbersome. 
Many faculty, including some in the DIST, regarded the process as an 
inordinately awkward device which had little potential for realizing 
utilitarian changes in the doctoral programs. These more emotive _ 
reactions certainly influenced 'substantive' reactions of the faculty 


to the process. 


Epilogue 


A summary or conclusion section is not appropriate here, because 
the evaluation was not completed. As discussed, the faculty groups 
delayed the process for six years. Recently, a new administration has 
assumed the leadership of the School and have, to date, been reluctant 
to pursue the process. The reader may, rightfully, ask, "For what 


purpose is the case offered?" The answer has two parts. 
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First, the reader has the basic rudiments of a curriculum 
evaluation plan designed to assess a complex doctoral program. It 
seemed to the author that there were not many 'live' examples of this 
type of plan in the literature. The reader has a rough, record of the 
substantive criticisms directed toward the plan with apunaprisite 
qualifications about the situational dynamics which may have influenced 
their criticisms. 

Second, the reader has a terse representation of faculty-administra- 
tion dialogue regarding the plan. The specifics of the situation are 
not generalizable in the technical sense of that word. It would appear, 
however, that the reader may profit from observing other professionals 
invest hundreds of hours in a series of activities that result in 
little or no observable effect on a doctoral program. 

These statements are not intended to suggest that the faculty 
of the DIST did not gain a more complete understanding of its complex 
doctoral program, or that the faculty may not have developed a more 
confident attitude about its strengths, or that the faculty did not 
develop a more articulate notion of its weaknesses. The author will 
advance the assertion that evaluation process as it developed in this 
situation clouded those real or potential understandings. There is 
now less of a probability that these understandings will be directed 
toward purposeful action than-if no evaluation had been planned or 
partially implemented. 

The author's remaining unanswered question is, "Was the faculty 
response concerning the complexity of the evaluation valid?" The 
decision to use formal criteria was insightful. The definition of 


the conceptual structure was clear and parsimonious. The operationali- 


web 


zation of the criteria logically followed both the structure and the 
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individual meaning of the criteria. The collective effect of the 


evaluation process was disabling in this circumstance. 
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be capable of a great deal of individualization. 


rest on sound student recruitment and selection 


provide for theory-practice integration. 


possess sophistication and rigor. 
be inquiry oriented. 
rest upon a quality faculty. 


be supported by adequate facilities, equipment, 


other resources. 


program should make explicit provision for self-renewal. 


TABLE 


1: CRITERIA FOR EVALUATION OF 


DOCTORAL PROGRAMS IN THE SCHOOL OF EDUCATION 


Daca to be collected Needed for 
Criteria 


——— 


List of all graduates of past five years with a 
description of present positions. 

A position paper describing the parameters of the 
area. 

A listing of common competencies of all doctoral 
graduates. 

A hierarchically ordered list of competencies which 
comprise the entire field. This list should be 
accompanied by the program experiences provided to 
achieve the competencies. 

Dossiers of all graduates of the past five years. 
Questionnaire to all graduates of the , ast five 
years, 

An analysis of student evaluation procedures. 
Description of student selection criteria; rationale 
for the selection criteria. 

Description of how selection criteria are operating. 
Description of student recruitment practices. 
Descriptions of development activities of faculty 
and students. 

Descriptions of practicum activities of the program 
area. 

Hierarchical listing of program area experiences. 
Description of inquiry oriented competencies of 
doctoral students. 

Descriptions of activities provided for student to 


apply inquiry skills. 
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Descriptions of inquiry oriented activities of 


faculty and students. 

Faculty activities in teaching, research, and 
service - also, included should be those who 
resigned and those who were offered positions 

but decided not to accept. 

Questionnaire to departmental.chairmen throughout 
the country evaluating the area faculty. 

Budget allowances for program area, 

Description of facilities. 


Description of the programs self evaluation plan. 


TABLE 2: A SUMMARY OF EVIDENCE REQUIRED. 


APPENDIX A 


CRITERIA FOR RATIFYING PROGRAMS LEADING 


TO THE DEGREE, DOCTOR OF EDUCATION 


December, 1972 


CRITERIA FOR RATIFYING PROGRAMS LEADING 


TO THE DEGREE, DOCTOR OF EDUCATION 


School of Education 
Indiana University 


Approved by Policy Council 
February 10, 1971 


At the present tine? there exist in the School of Education of Indiana 
University 20 programs leading to the degree, Doctor of Fducation. Most of 
these programs have been in existence for a number of decades, and a sub- 
stantial time has passed since their original approval. During that interim 
the world of professional education has changed markedly, as even a casual 
observer of the scene can attest. It is likely that over the next decade 
circumstances and conditions will alter even more dramatically. In the face 
of this challenge, the School of Education must look to the quality, rele- 
vance, and flexibility of the programs it offers. At the undergraduate level, 
and to some extent at the Fifth Year level, reviews and reformulations are 
already under way through the mechanism of the Commission on Teacher Education. 
At the Fifth Year level, moreover, all departments and program areas are 
currently redefining their core requirements. Similar reviews are mandated 
at the doctoral level, and will be carried out by the Advanced Graduate 
Studies Committee commencing during the present academic year. The purpose 
of this documeat is to set forth the criteria that will be used by that 
Committee in~assessing the quality, relevance, and flexibility of current 
programs as well as any others that may be proposed. 


The statement is divided into four parts. Part I deals with certain 
criteria that are frequently but erroneously cited as relevant to doctoral 
programs; the purpose of the part is to lay these to rest. Part II deals with 
the purposes of doctoral programs and provides a general base for interpreting 
the criteria to be proposed in Part IV. Part III is concerned with drawing 
a distinction between substantive and formal criteria, and argues that only 


formal criteria are appropiiate for a general review of the sort to be made 
by the Advanced Graduate Studies Committee. Finally in Part IV, eleven 


formal criteria are proposed and explicated which will serve as the basis for 
the Committee's review. 


I. Some Invalid Criteria 


There exist certain criteria which are frequently cited as having 
validity for judging the suitability of doctoral programs but which are in 
fact as experience has shown, infalid for that purpose. These criteria are 
vriefly mentioned below and an argument is made against their validity. 


1. Professionalism. Doctoral programs by long tradition are said to 
eschew professionalism. ‘Professionalism in this sense refers to a kind of 
vocationalisin where the pruposes of the vocation differ from the purpose of 
the university to advance knowledge. For example, the purpose of the vocation 
"philosopher" is tne advancement of knowledge and so a program in philosophy 
is not considered to be a professional progran; while the purpose of the 
vocation ‘teacher' is not the advancement of knowledge and so a teacher 
education program is considered to be a professional program. 
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There are two protlens with this approach. First, it confuses lower 
level training of practitioners with higher level training of students of 
professional problems. Thus, there is no doubt that the training of elemen- 
tary or secondary teachers, of accountants or marketing executives, of chemical 
laboratory technicians, or of United Nations foreign language translators is 
professional training in this sense, but it does not follow therefore that 
the training of students of the learning process, of finance, of research 
chemistry, or of linguistics is also professional training simply because it 
happens to take place within the identical institutional setting (i.e., a 
given department or school) or under the guidance of the identical facult} 
members. One ought not, that is, consider a doctoral program in a school 
which has a teacher training program to itself be a teacher training pro- 
gram by virtue of that contiwuity. 


There is a second difficulty with this insistence on separating 
pursuit of knowledge as a professional goal from pursuit of some practical 
vocation, and that is, that all doctoral programs do prepare a candidate, 
however effectively or ineffectively, for some career. In this sense all 
graduate departments can be arranged along some continuum of professional- 
ism. There is no doubt that the chemistry candidate is preparing to be a 
professional chemist; that the sociology candidate is preparing to be a 
professional sociologist; that the economics candidate is preparing to be 
a professional economist, just as it is clear that the education candidate 
is preparing to be a professional educationist. The rigorous application 
of a non-professionalism criterion would result in the rejection of almost 
every doctoral program in existence. 


Thus it seems reasonable to assert that the criterion of professional- 
ism is a false one with which an emergent graduate school ought not to be 
concerned. 


2. Pure knowledge. Just as they are traditionally not to be 
concerned with professionalism, doctoral programs are not to deal in applied 
knowledge. The pure-applied dichotomy is frequently drawn; it is asserted 
that a graduate department should deal only in pure knowledge and should see 
to it that doctoral dissertations are genuine contributions to knowledge. 


This argument is patently specious. In the first place, the distinc- 
tion between pure and applied knowledge is not absolute; if it is worth draw- 
ing at all it is only as theoretical ends of a knowledge continuum. It is 
likely that no knowledge is in fact representative of the farthest ends of 
this continuum; that, on the contrary, almost all knowledge occupies an 
intermediate position. Furthermore, there is an enormous interaction between 
these two varieties of knowledce so that it is in fact impossible to pursue 
one without involving the other (or nearly so; consider the nearly two millenia 
required by mathematicians to "free" the assumptions of Euclidian geometry 
from real world physical referrents). 


A recent effort in the social sciences germane to this problew has 
been the attempt by Cronbach and others to distinguish "conclusion-oriented" 
from ‘decision-oriented' inquiry. The former presumably leads to relatively 
more ‘pure’ knowledge and the latter to relatively more "applied" knowledge. 
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3. 


This analytic distinction has prove?! useful in resolving sone of the differ- 
ences among classic and emergent theoreticians over the proper methodology 

for e\iluation and other accountability processes. While one can argue over 
the ex. t degree of validity of the formulation, the mere fact of its existence 
does indicate that the issue of pure vs. applied knowledge is by no means 
resolved in the minds of even leading thinkers. If this be the case, its use 
as a criterion for judging doctoral programs is surely questionable. 


3. Objectivity. Every discipline has its own methods, and one of 
the objects of doctoral level training is to produce a practitioner of the 
discipline who can use those methods in a completely impersonal way. It 
is argued that no discipline can be a true discipline unless its adherents 
can produce essentially similar findings in its pursuit. Hence every man 
must make himself completely submissive to the discipline (indeed, this 
fact may be the major justification of even calling an area by the term 
“discipline") if he is to be an acceptable practitioner, well regarded by 
his colleagues, and trustworthy to induct others. The man who wishes, in 
the common parlance of the day, to “do his own thing," essentially a mono- 
lith, uncompromisiny in its demeanor, which exacts conformity from every 
candidate, at least until he proves himself competent and reliable. 


Such a position is hardly compelling. If gains in knowledge were in 
fact made only incrementaily, with each bit patiently- and painstakingly 
fashioned so as to fit onto what had gone before, this position might be 
maintained with greater conviction. But many gains, and certainly all 
break-through gains, are not made in this way; history is full of examples 
to the contrary. Indeed, the university has accepted the essential soundness 
of the inverse position by its arguments for academic freedom, which is de- 
signed to allow a man to pursue truth wherever it may lead him and to propose 
ideas without fear of repression or derision. If doctoral programs are to 
take full advantage of the normal processes by which progress is achieved, 
they must be flexible, open, offering alternatives, and nuturant of individ- 
ual differences in point of view and interest. Jancks and Riesman have 
referred to the need for "more mobility and anarchy" in graduate programs, 
and for the legitimation of more flexible and more cross-disciplinary pro- 
cesses. This is a call which the emergent graduate school can ignore only 
at the risk of its own demise. 


4. Onnicapability. Present doctoral programs are typically designed 
to turn out interchangeable parts--professionals who can take up any duties 
that are properly within the domain of the discipline and carry them out with 
competence and skill. In part this is the result of faculty porkbarreling, 
with each specialist having his turn at the graduate students and allowing 
everyone else his due; and in part the result of a sincere desire to train 
a ‘well-rounded’ individual. Thus, all candidates are exposed to all aspects 
of the discipline and become equally competent (or incompetent) at all, save 
only that area which, by serving as a focus for the dissertation, receives 
somewhat more detailed attention. 


But this situtation is hardly desirable. First of all, it ignores 


the fact that different interests, different aptitutdes, and different compe- 
tencies. Second, it ignores the fact that different candidates will pursue 
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different career lines that require somewhat different training. Finally, 
it ignores the rapid changes in knowledge that produce more and more 
specialization and that tend to turn traditional discipline areas into a 
series of only distantly related sub-specialities (indeed, it is often the 
case that communication is easier between incumbents of new sub-specialties 
of different disciplines that between incumbents of a sub-specialty and a 
parent discipline). All of these factors call for differential training mad 
a rejection of a program that manufactures identical products. 


Il. Purposes of a Doctoral Program 


The preceding section attempted to lay to rest certain criteria 
that are not applicable to a doctoral program. Before proceeding to a 
discussion of applicable criteria, it will be useful to consider briefly 
what the purposes of a doctoral program are. In the absence of a statement 
of purposes it will be difficult to judge the appropriateness of any 


criterion that might be proposed. These purposes include at least the 
following: 


1. To advance knowledge and to stimulate inquiry. Probably the 
most general purpose of a doctoral program is to extend knowledge, particularly 
through inquiry. The dissertation is not merely a hurdle which a student 
must be able to jump; it is intended to be a genuine contribution to extend- 
ing the knowledge frontier. This is not an argument for "pure" knowledge, 
as that term was used earlier, but rather for focussing on that part of the 
world and its activities which are the peculiar domain of the academic. 
Indeed, to a very great extent, universities are established by society for 
the purpose of generating knowledge, and doctoral programs are a key part 
of that general goal. Criteria that are proposed must therefore deal 
explicitly with this purpose. 


2. To impart certain hiph level competencies and skills. The doctoral 
program is intended to bring the candidate to the frontiers of knowledge in 


a given discipline, to bring him into the vanguard of inquirers and workers. 
The competencies and skills that are required to bring him to this level 
are both substantive and methodological, and it is in their mastery that a 
candidate can claim privileged membership in a professional group. The 
criteria must therefore be concerned with providing for this content and 
methods mastery. 


3. To reproduce members of the profession or discipline area. The 
doctoral program is the final training ground in which menbers of a profession 
or discipline area can reproduce themselves. Taking this purpose into account 
in setting criteria means asking whetner proposed criteria do in fact stimu- 
late this reproduction on the one hand, and whether they militate against 
the reporduction of undesirable characteristics, on the other. The concern 
must be, so to speak, with whether the criteria deal adequately with the 
"genetics" of tue situation. : 


4. To license or credential professional activity. Professional 
activity takes place within some organizational and social context. One 
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purpose of grantiny a decree is to provide assurance that the impact will 
be positive and within ethical limits. Any proposed criteria must be 
concerned with affording protection against fraudulent practice, both in 


relation to other menbers of the profession or discipline area and to 
persons outsice that area. 


III. A Distinction Between Formal and Substantive Criteria 


Generally speaking, criteria can be cast in one of two modes: they 
can be formal, i.e., dealing with the form of extrinsic aspect of the thing 
to be judged or with the conventions covering its use; or they can be 
substantive, dealing with the content or intrinsic aspect. The latter 
judgments can be wade only by someone who is familiar with the content, for 
obvious reasons, and in the case of a doctoral program, that clearly means 
the faculty members who inhabit the department whose graduate program it is. 
This fact produces an immediate problem, however, for if the best substantive 
judgments can be rendered only by the specialists in the discipline, how 
can one be sure that the judgments are objective? Having proffered a pro- 
gram, can a department find it anything else than valid? 


This is a serious dilerma, and one that is constantly confronted by 
any group, sucn as the Advanced Graduate Studies Committee, that is 
attempting to render quality judgments about the substantive aspects of the 
programs. Surely that Committee, or any other similar body, would not claim 
omniscience for itself; it is very likely to make “mistakes” unless some 
mechanism is used to guard against them. 


One wey to deal with the problem is simply to ignore it. One can 
admit its presence, and then say, ‘But every department that comes before 
us is subject to this same difficulty," as though this equal applicantion 
of inequity excuses its intrusion. If the mistakes made by an inexpert 
group in renderine quality judgments were systematicaly biased; this argu- 
ment might make sone sense, for everyone would be equally disadvantaged 
(or advantaged, depending on the nature of the bias). But the errors, 
because they are based on ignorance, are much more likely to be randon, 
so that one department is advantaged while another is disadvantaged. 
Inequities of this sort simply cannot be tolerated. 


Another way to handle this problem is to utilize reviewers from 
related areas. Thus, a physics program might be reviewed by persons from 
chemistry, astronomy, and mathematics: or a psychology department by 
persons from sociology, anthropology, and political science. But the 
undeniable fact that their expertise in the area to be judged is tangential 
and limited males it possible to raise questions about the judgments no 
matter how valid they may in fact be. ‘Moreover, simple equity requires 
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that if there is to be a nezative decision it ought to be made by persons 
who are at least as expert as those whose program is being denied approval. 


A third and most saticfactory way to handle this problem is to 
incorporate the judgments of persons from identical departments outside the 
University, at least in questionable cases. This can be done in two ways: 
by a direct outside evaluation, oc by an "audit" of inside evaluation 
procedures and conclusions by an outside group. Such a group is in the 
best possible position to answer questions about program quality and faculty 
quality and while their judgment may still be questioned, it is at least 


as valid and undoubtedly more objective than that rendered by the department 
itself, 


Given this dilemma (anda whatever may be the method adopted to deal 
with it), it is at once apparent that it would make no sense to attempt 
to generate general substantive criteria for judging all doctoral programs. . 
If substantive judgments depend upon an intimate and detailed knowledge of 
the field being judged, general criteria are impossible. Hence, it is only 
with formal criteria that this report is concerned. 


IV. Some Formal Criteria 


The criteria below are formal criteria offered as consistent with 
the point of view that has been espoused in the preceding sections. It 
is stressed that this listing is neither final nor complete, and should be 
subject to constant criticism, refinement, and extension. 


‘ 


lprograns in the School of Education stand in a peculiar relation- 
ship to this dilemma because it may be argued that many of them, e.g., 
educational psychology, are bi-dimensional, cutting across education and 
some other well-established discipline area. ilence, persons from those 
other disciplines are in a particularly good posture to render quality 
judgments. But this position argues for a kind of reductionism that may 
not be valid. This position (often called the "foundations approach to 
education") has Leen sharply critiqued by numbers of authors who have 
been concerned with the question of whether or not education is a unique 
science or discipline. Prominent among these writers is E. S. Naccia of 
the Department of History and Philosophy of the School of Education. The 
contention of these writers, based upon close philosophic analysis, is 
that education probably can be characterized as a separate science or 
discipline, and that at worst, one may revsard this as an open question 
subject or empircial examination. On that sround, it may be mistaken 
simply to assune, without further evidence, that the bi-dimensionality 
of educational areas is real, or that the judgment of persons from 
apparently related areas has any wore validity for education that it 
would have in other program areas. 


1. The program should display an integrity or wholeness.* It 
must be nore than simply a random collection of components: it must 
add up to something. But the requirement of adding up should not be 
inflexibly interpreted--it may add up to several things at once or to 
different times. ilor does this criterion imply a strict autonomy for 
an area in the sense that a program cannot depend for certain of its 


components upon other discipline areas (indeed, interdisciplinary 
dependencies are to be encouraged). 


There are probably multiple ways in which such integrity can be 
demonstrated. One way is that a candidate, having proceeded through a 
program, will find particular employment opportunities awaiting him; 
his program has added up to something that the labor market recognizes. 
Another way is to show that common conceptual strands up through the 
various program elements. Yet another way is to relate the program to 
certain common phenomenon, as for example, when seemingly disparate 
work in sociology, linguistics, curriculum, economics, and health can 
all be shown to have relevance to inner city school problems. No doubt 
many other ways could be identified, but these examples will suffice to 
show that program integrity is readily demonstrable in a formal sense. 


2. The program should be tnquiry oriented. ‘The hallmark of the 
well-trained doctoral student is an inquiring mind. Inquiry must there- 
fore be at the heart of all training programs. 


This statement should not be interpreted to mean that only pure 
knowledge should be the object of the inquiry. Efforts have recently 
been made to distinguish conclusion-oriented (that is, theoretical) from 
decision-oriented (that is, practical) inquiry. This is clearly a most 
useful distinction. It is patent that both forms of inquiry have a place 
in a graduate department, particularly if the decision to be serviced by 
decision-oriented inquiry is of sufficient importance. An interesting 
example is the present concern with ecology and environment; surely no 
graduate department vould turn away from environmental inquiry simply 
because it is declsion-oriented, 
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“The concept of integrity should not be interpreted as equivalent 
to saying that programs siould represent traditional discipline areas. 
Tnese traditional areas have been more or less arbitrarily defined, or 
the definitions have euerged largely as historical accidents. There are 
many other ways in \nich integrity can be demonstrated. So for example, 
the integrity of African Studies Programs is ell understood, even though 
such prograus may cut across traditional disciplinary lines such as 
economics, georraphy, sociolozy, and the like. 


here are several useful indicators of whether or not a graduate 
department is in fact oriented. On would expect first that inquiry would 
ve prized vy the faculty, ‘no vould show their comaitment through their 
own scuolarly inquicy. Faculty writin',s would probably best demonstrate 
this cormmituent. further, one sould expect that students would be 
trained in metuods of inguiry, whether explicitly, as they often are in 
social science departments, or implicitly, as they typically are in 
physical science departments. ‘The effects of this training should be 
evident in doctoral dissertations, Finally, an inquiry orientation implies 
the existence of a substantial body of literature which can be tested, 
refined, or applied by virtue of faculty/student inauiry. 


3. Tne program should possess sophistication and rigor. Obviously 
what can be expected in tenis of this criterion depends to a very great 
extent on tie decree of evolution of the discipline area in question, and 
therefore a julyment of sophistication and rigor comes close to being a 
substantive rather than a forinal one. Sut what is meant here is capable 

of judgment in a formal sense. Included are the concepts of comprehensive- 
ness, of scope, and of difficulty. 


A doctoral progran should certainly be pitched at a level which 
would be beyond the typical undercraduate student. So for example, the 
reading required ought to be substantially different from that which 
appears on typical undergraduate bibliograpnies. The pattern of progress 
throuzh the program ought to be guided by sensible prerequisites which 
permit building up of difficulty and continuity as prosress is made. 

The program ought to move into the frontiers of knowledge and serve as 

a useful heuristic for tne candidate in guiding his inquiry as he moves 
into a full professional role. The program should reflect the most 

recent advances in the area and related areas. ilore than just learning or 
facts should be involved, and tueory and mettods should have a prominent 
role. Finally, the program should be related to a substantial and reason~ 
able unique Lody of literature which is to be extended or applied, and 
vhich is tae corpus of the professional knowledge in the area. 


4. “he prorran should provide for theory-practice intepration, 
The provleus of professionalis:. auu pure knowleare have already been 
dealt with. itather than to claim that it is essentially non-professional 
and non-appli2c, each sraduate department ougut to take specific steps to 
see to it that the lLnovledge which it generates can have practical sienifi- 
cance. It is aptly said taat if practice without theory is blind, theory 
witnout practice is empty. But such an expression is essentially meaning- 
less unless specific steps are taken to bridce the gap, the mere expression 
of a pious hope is that direction is not sufficient. 


Probably the best test for this criterion is the presence of 
development activity in a department which is particularly ained at 
translating theory into practice and of reflecting practice to the 
theoretician ty way of feedback. Specific examples of the way in which 
theory is translated into practically useful applications should be avail- 
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able. These might, for example, include packages of training materials, 
measurement instruments, organizational patterns, or other innovative 
applications, The relevance of theory for the real world can be con- 
stantly validated by these means. 


5. The program should be competency-based. Another way of saying 
this is that all program requirements should be related to a set of 
competencies which have been defined as appropriate to a candidate in 
the area. Wor all competencies need to be required’ of all students, there 
may for example. be a core of competencies to be mastered by all and other 
more specialized competencies from among which individual students may 
select as befits their interests or their career aspirations. 


To meet this criterion a program area ought to be able to point 
to a listing of competencies, derived through an empirical study of what 
professionals in the area actually do; through conceptualization of what, 
under ideal circumstances, they might do; through study of the literature 
dealing with the area; and/or in other ways. The area should be able to 
show that the competencies were derived by some rational process and 
that various program elenents and requirements have been analyzed to 
determine their particular contribution to these competencies. Compe- 
tencies should, moreover, be defined in performance terms, and examina- 
tions and other evaluative procedures should be designated to permit the 
candidate to demonstrate that he has mastered the competency. Finally, 
the set of competencies should be ordered and articulated to provide the 
integrity alluded to in the first criterion above. 


6. The program should provide multiple experiences that permit 
achievement of every competency that is required. These experiences may 
be of many sorts: didactic, vicarious (as in a film), laboratory (a kind 
of real-life simulation), or direct (as in an internship). Each experience 
must be appropriate to a particular competency (or set of competencies). The 
choice of which experiences to be undertaken to achieve each competancy 
should be left to the student under the guidance of his major professor. 


The test of whether this criterion is met is simply the existence 
of such rmiultiple experiences clearly defined, and the possiblity that a 
student may devise multiple routes through the program by selecting from 
different experiences to meet requirements. 


7. The program should be capable of a great deal of individual- 
ization. This criterion in part follows from the preceding one, but is 
dictated mainly by the desire to avoid the problem of omnicapability. 

Tach student ought to be able to select what seems to hin (with guidance) 

to be an appropriate pattern of learning, leading to a somewhat different 
specialization from that of any otler student. The making of such selections 
implies a systematic student evaluation procedure that permits determining 
the status and emergent needs of a student at any given time. All that 
should b2 required of him is that in making his selection he preserve the 
integrity called for in the first criterion above. 


10. 


There are several possible tests for this criterion. First of all, 
the area should be edie to display a variety of program options from among 
waich selection can be made by a student. The area should be able to 
show that different selections can sensibly lead to assumption of different 
professional responsibilities. They should be able to demonstrate that 
students are encouragec to puruse, and have purused, different program 
asseublases. They should be able to point to concrete cases of graduates 


who differ notably from one another in the things they have been prepared 
to do and feel able to do. 


8. The program should rest on sound student recruitment _and 
selection criteria. It is a truism that a good prograt attracts good 
students. It is equally true that a program which is less than superior 
will cease to attract good students. While a sound reaction in this 
situation would be to reform the program, a wore typical reaction is to 
reformulate selection criteria. lo program area is likely to engage in 
self-liquidation; a reductiou in the number of students is almost certain 
to be countered by an easing of entrance requirements. Thus, the level 
and quality of recruitmer.t procedures and selection criteria is a good 
index of program quality. 


A first indicator of whether this criterion is met is thus the 
existence of selection criteria. This should not be interpreted inflex- 
ibly, however. pvifferent departments may utilize different criteria 
based on their own needs (e.g., a portfolio of art works would be an 
excellent selection base for art education but not elsewhere), and the 
same departuent may have different criteria for different kinds of 
students (e.g., recruits from culturally deprived populations). But a 
good program will have sone means of assessing student aptitude (e.g., 
via the Graduate Record Examination or the Miller Analogies Test), and 
of determining whether the student prossesses an adequate knowledge 
based to enter rigorous doctoral training (e.g., by some process of 
determininy amount and quality of past training and experience). The 
sood program area will also avoid what has been aptly called “doctoral 
pollution, i.e., the overproduction of doctorates in areas or along 
career lines that are already overmanned; instead, the program area will 
take advantage of a buyer's warket to raise its standards or to explore 
new avenues of training. 


9. ‘The prorranm sliould rest upon a quality faculty. What consti- 
tutes a quality faculty is, of course, largely a substantive matter and 
hence beyond the pale of this paper. But there are certain formal matters 
that can be attended to that will provide some index of quality. 


Certainly the nature and source of a faculty's degrees can be 
detert:ined, as can the degree of institutional inbreading. The number 
and nature of a faculty's publication is easily assessable, as is the 
extent to vhicn their writing is influeatial throughout the profession 
(measured, for exaniple, by tne number of tine publications are quoted, 
revieved, included in reference lists, etc.). In general, many of the 
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same criteria that are applied in determining whether a faculty member 
should be promoted or placed on tenure may also be applied here, Finally, 
and perhaps most importantly, the nature of the faculty reward system in 

the program area can be investigated. Are faculty members seemingly 
rewarded for conformity or for divergence? Are faculty encouraged to 

relate to tueir professional reference group or are they expected to 
maintain a perspective largely within the university? Are {nterdisciplinary 
contacts encouraged? And so on, 


10. The prograa should be supported by adequate facilities, equip- 

A program that lacks adequate support mechanisms 
cannot be effective regardless of how noble it may be in concept. Over- 
crowded spaces, vintage laboratory equipment, outdated instructional 
materials, low salaries, and the like, can only detract from efficacy. 

While these are primarily administrative considerations they are neverthe- 
less crucial. It may also be noted that the lack of adequate support in a 
situation where support is on the average reasonably good is a fine indicator 
of lack of administrative confidence in an area. 


ll. The program should make explicit provision for self-renewal. 
Simply, che program must nave a means for assessing its status at any given 
time and for comparing that status to some ideal. The ideal itself must 
be subject to revision, and indeed, a department that had not revised its 
ideal given the events of, say the past decade, would be in a dubious 
posture. ilowever good a program mey be, it is always possible to improve 
it, and at times it may be wise to abandon it. The department that does 
not make explicit provision for reassessment is likely to forget the 
matter entirely, or to overlook it at just those times when rigorous 
assessment is most needed. 


The test of the criterion is the existence of a plan for such self- 
evaluation and some historical evidence to show changes that have been made 
over tine in response to what the self-studies showed. 


A Final Word 


It is obvious that the criteria named above are very broad and are 
intended to apply to an entire program over time, not to pieces of it or 
to a one-tine assessment. The academic health of a doctoral program can 
no more be determined by looking only at its courses, or its admission 
requirements, or its examination svsten, or to the placement of its graduates, 
than can the health of a man be assessed by looking only at his liver or 
his vision. Nor is it the case that a diseased organ noted at a given time 
will later be diseased as well, or that the disease of any part necessarily 
means the demise of the whole. Any review procedure that attempts to apply 
the above criteria ought not to begin on the assumption that any program 
that does not conform should be disapproved; rather, one should exp2ct 
all prograus to be partly deficient in iiany areas and pernaps seriously 
deficient in some. The object is not to bring recalcitrants into line 
but to help all programs do better. What is required is a therapeutic 
rather than a judsmental point of view; evaluators should be perveived at 
least as much as physicians as they are as judges. 
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APPENDIX B 


INSTRUMENTATION ASSOCIATED WITH THE CRITERIA 


Criteria “uestion 1: Does the program display an integrity or wholeness? 


A. Evidence Required: 

1. Professional employment for which student candidates are eligi- 
ble. 

Descriptions of how various courses or other program experiences 
can be combined in a doctoral program to equip a person for a 
particular type of position. 

A description of the common conceptual strands in the program 
courses and activities. 

4. A description of the core knowledge and skills of the. program. 

B. Suggested Procedures for Data collection: 

1. The program area should supply a list of names of all doctoral 
graduates for the past five years. Accompanying each name should 
be a description of the graduate's present position. This de~ 
scription should indicate the major professional duties and ac- 
tivities, i.e., research or evaluation studies, program develop- 
ment activities, and other service activities. A format for col- 


lecting this data follows: 


lame of Graduate 2. Present Position 3. Where employed 
John Smith Assistant Professor Soho College 


Professional Duties and Activities 5. Activities in Graduate Study 
Provided by the Program Area 

Teaching two undergraduate ed. psych 

courses; continuing work begun as 

doctoral study (the relation of course work - especially exper- 

critical reading to divergent think- imental research design; in- 

ing); consultant to nearby headstart structor of P280 courses; work 

program; working on a faculty-student on the effectiveness of headstart 

committee to revise fifth year ed. program. 

psych. offering. 
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2. A-postion paper. should he. developed by the program area which 
describes the parameters of that area. This paper should include 
a@ listing of the conceptual strands on which the program area is 
based and the assumptions underlying the inclusion of these con- 
ceptua}. strands, mi 
3. The program area should provide a list. of competencies or com- | 
binations of possible competencies (knowledge and/or skills) 
which are common to all- doctoral graduates, in that area. (This 
information is also being gathered for criteria, question 2, B, 3) 
C. Performance Jriteria 
1, For most graduates there should be some relations between their. 
graduate program activities and their present position. 
2. The position, paper should clearly.describe the program area-.and 
differentiate it from other areas. 
3. The list of the common competencies ehould. be available or should 
be developed. .... a 
Criteria Question 2: Is the program competency-based: and-does the pro- . 
gram provide multiple experience that permit achievement of every: 
competency that is required? 
A. evidence Required: 
1. A list of competencies which comprise ‘the entire field. 
a. The list should be hierarchically ordered. . 
b. The list should be stated in behavioral terms. 
2. A list of.the kinds of experiences provided in the program to. 


“make competency-achievement possible. 


3. A list of means of evaluation in’ the program of each compe-' 
tency. 

4. A rationale for competencies for which no experiences are pro- 
vided in the program because the competencies are not considered 
important enough to warrant them. 

B. Suggested Procedures for Data Collection: 

1, Two alternative methods may be used to provide this competency 

list: 

a. If the professional organization of the field already has 
a list of competencies and it is accepted nationally as a 
standard, this can be used. 

b. If not, the procedure should be to devise a list of all 
known competencies in the field (working from literature and other 
such sources), and then submit this list (behaviorally stated) 
for rating in order of importance to a panel of the most vis- 
ible leaders of the field (minimum: 40). This questionnaire 
should also provide a place for these leaders to add any' other 
competencies tiiat they feel are equally desirable. 

2. After the questionnaire have been returned, and ratings have been 
checked, a new competency list should be reassembled on the basis 
of the responses. The hierarchical order will'thus have been’ 
established. (It is assumed that all competencies will have been 
written in behavioral terms before being included on the question- 
naire and that any additional suggestions forthcoming will be re- 


stated in behavioral terms if not already in that form.) 
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3. The new competency list would indicate the various kinds of ex- 
periences provided in the program for its achievement. This 
list would include the name and/or number of the course provid- 
ing the experience and a brief statement of how it is assessed. 


A suggested format follows: 


) 


Kinds of Experience Provided 
Competency Didactic Types Vicarious “ Labor’ © ~"Direct 


Item Description; course 
name and/or number; 
where and how as- 
sessed, 

Performance Criteria: 


1, The program should evidence, through the results of the compe- 


tency list, that it is competency based and provides multiple 
experiences for competency achievement. ; 
Criteria Question 3: Is the program capable of a great deal of individ- 

ualization? 

Evidence Required: 

1. A student may devise multiple routes through the program by 


selecting form different experiences to meet requirements.. 


Students are pursuing different program assemblages. | 


Cases of graduates who differ notable in the things they 


have been prepared to do and are able ta do. 
4. A systematic evaluation procedure which, determines the status 


and emergent needs of students at any given time. 


See 


B. Suggested Procedures for Data Jollection: 


l. 


Dossiers of all graduates for the past five years would be used 


to ascertain if: 


a. Students devised multiple routes through the program; 


b. Students pursued different program assemblies (different 
minors, outside work, etc.) 
The competency list (criteria Question 2, B, 3) should be sent 
to all graduates for the past five years. The graduates should 
check those competencies (a) he was prepared to do by the pro- 
gram, (b) he feels able to do, (c) in order of importance, and 
(d) he utilizes in his present position. 
The program should provide an analysis of its student evaluation 
procedures as they are currently operating. The aim should be 
to establish that they are systematic in nature and function 
efficiently for all students in the program. Details of the dif- 


ferent kinds of activities should be included. 


Cc. Performance criteria: 


l. 


3. 


Students have devised and are devising multiple routes through 
the program. 

Graduates are utilizing the competencies they were prepared for 
by the program. 


jvaluation programs are being used. 


Criteria Question 4: Does the program 


and selection criteria? 


A. Evidence Required: 

1. A description of the selection criteria used by the program 
area, which should include assessment of general aptitude, 
specific aptitude, achievement, etc.; and the rational for 
the use of these assesement techniques. 

2. Descriptions of how the recruitment program reflects the demands 
of the market place. 

B. Suggested Procedures for Data Collection 

1, A rational argument should be presented for the use of the selec- 
tion criteria. 

Test scores, achievement records, quality and quantity of pre- 
vious training, and experience should be listed for ali students 


admitted during the previous five years. ~ 


Graduate performance of students in the program should be pro- 


vided. 
A description of ‘the first position the student obtained after 
receiving the doctorate should ba provided, 
Test scores, etc. for the students who were admitted ‘but who ~’ 
did not matriculate should be provided. 
Documentation of the demands of the market place (job vacancies, 
national conventions, etc.) should be provided. 
A description of national recruitment procedures should be pro- 
vided. 
C. Performance criteria: 
1. A very high percent (90%) of the students who were admitted to 


the program area actually matriculated, completed a degree and 


“9s 


were placed in a respectable position. The new position re- 
flects a marked change in responsibility for most doctoral 


recipients. 


Criteria Question 5: Does the program provide for theory-practice in- 
tegration? 
Evidence Required: 
1. A description of the development activities of faculty and stu- 
dents. 
2. A description of practicum activities of the peowtinn area. 


B. Suggested Procedure for Data Collection: 


1. Documentation and descriptions should be provided of development 


activities engaged in by faculty, studants, and especially 
faculty-student teams; this description should indicate how the 
development activity is related to some theoretical base. 
2. Documentation of practicum activities should also include a 
description of how they are related to some theoretical base. 
Performance Criteria: 
1. The program area should be involved in development activities which 
are built on theoretical bases. 
Criteria Question 6: Does the program possess sophistication and rigor? 
A. Evidence Required: 


1. A description of program area experiences which indicates the 


increased sophistication demanded at advanced levels. 
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B. Suggested Procedures for Data Collection: 
1. The program area should provide.a hierarchical listing, of, area 
experience in the following format: 
Area Activities: courses, Student Competencies Append bibliograph- 
practicums, independent Developed in Activity tes, which describe 
study, etc. (Activities . ‘ extent of literature 


should be listed from basic for this activity. 
to advanced levels.) 7 


c. Performance Criteria: 


1. The listing of activities, described by competencies aad evidence 
of extent of literature, should Seananre an increased level of 
sophistication at advanced fovea, 

Criteria Question 7: Is the program silage ortereaay San 
A. Evidence Required: _ | o 

1. Descriptions of the inquiry competencies faquired of doctoral 
students in the program area. 

2. Descriptions of the planned learning expevtendes for doctoral 
students to develop and apply their jhanire ekélin, ” 

35 Descriptions of inquiry oriented ccpivities at the faculty — 
and students. . ; 

B. Suggested Proestupes for Data Collection: 

1. Competencies or combinations ot Sonpetencie’ in the inquiry 
area which are required ‘if all doctozal students should be 
listed. * agentes 


2. Descriptions should be provided of all inquiry oriented activities , 


which are provided in the program area. 


x 


1. 


2. 


9 - 


The program area should provide a listing of the inquiry oriented 
activities of faculty and students for the past three years, which 
would include publications, evaluation service activities, artis- 
tic exhibitions, etc. ‘The program area should also collect, by 
means of a questionnaire, a list of the inquiry oriented activi- 


ties of the doctoral program graduates of the past five years. 


Cc. Performance Criteria: 


The program area should provide experience for doctoral students 


to anply their inquiry silis. 
The competency list should provide evidence that the program area 


is inquiry oriented. 


Criteria Question 8: Does the program rest us0n a quality faculty? 


f, Evidence Required: 


1 


“+ 


Sections of the faculty annual report/questionnaire which re‘ate 
to the teaching effectiveness, research productivity, and ser- 
vices rendered, 

Informetion on che fecu.ty who have resigneé during the previous 
five years, the information providing a basis for assessment of 
the resicgnex. 

Information on the facu:ty vho vere offered a position but de- 
clined during the past five years, the information providing a 


basis for assessuent of che person, 


B. jJuggestec Procecure for vata Colitection: 


The information should be outlined to vef.ect faculty productivity, 
teaching ana service. 

The quality of the facu.cy should be documented so that an assess- 
ment regarding its quaiity can be estab’ished, i.e., depmean he:d, 


rate of xxc.otion and other rewercs anc recognitions. 


4% 


3. 


/ questionnaire siould be sent to a representative sample of de- 


partmental cheirmen throuchout the United States. The following 


is a susgested format for the questionraixe: 


f. 


Which of the terms below best describes your judgment of 

the quality of bi graduate faculty in your davaebrut at 
Indiana University: Cotidider sale the scholariy competency 
and achievements of the present facuity. 

l. "Heeineutehed. 

2. Stronc. 

3. Good, 

4. Adequate. 

5. Marginal. 

6. Not sufficient to provide accertabie coctoral training. 
Insufficient information. 

Hot) vould you rate the Indiana rik vensety School of Education's 
oevariment of if you vere selecting a 
gracuate school to wort: for a doctorate in your field today’ 
Consider the eccessibiiity of facuity end their scholarly . 
comzentence, curricuium, educational and research facilities, 
the qua.ity of graduate students and other Factors eich gene 
tribute to the effectiveness of the doctoral snotty 

.. Extreme:y attractive. 

- Atcractive. 

33 heceptad-e. 


4. Not attractive. 


5. Insufficient information. 


Performance Criterte: 
The facuity suoule be casity recotni.ed across cepartments within 
the universicy end vichin their sreas of sseciaii:ation on s 
nationa! basic. 
The attraction and retention rate of the facuity should reflect 
an improvement in their quarity. 

Criteria Cveation 9: Is the program supported by adequate facilities, 

equinment, and other resources? 

Evidence Required: 
Tne budset for che denertment or program area for the previous 


five years should be nroviced. ‘his vould include faculty 


salaries, f£ salaries, suppiies end expenses and new equin- 


ment. Increases in the allocetions for each of these sub-. 
cetecsories should be sho:n. 
b. Suggested Procecures for Tata Col ection: 

Tne atloccted budsets should be presented so chat comparisons 
can be made <cross five yeer period. Each of the major cate- 
geries of the buc;et should be itemi:ec. 

2. <A description of office snaces and facilities shouid be giver. 

3. A Gescrintion of faci.ities outside enc inside the school and 
departnents shouic be proviced. 

Performence Criteria: 
Tacu..ty showid have cometitive sa.cries. 
Secretory curport shouic be competitive. 


Support sucalc be coimpeticive. 
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Criteria Guestion 70: Does the sropram male exnticic nrovision for self- 


renetva:.’ 


Evicence Requireu: 


1. A Gesien for seif-eve uction. 


5 
vj 
a 


The resu.ts of the most recent self-study, including recommenced 
changes. 


wecested Procedures for Data Collection: 


1, An outline of the recommended chances resulting from the most 


recent se.f-stucy shoulc be given. 


2. 


Tecumentaticn of the chanzes which have been made or a pian for 


mal:ing the recowmencec changes should be provided. 


Performance Criter‘a: 


t least one-se.f-study resu’%ted in significant changes during 


tne previous five 


APPENDIX C 


PERT CHARTS DESIGNED TO GUIDE THE DIST 


IMPLEMENTATION OF THE EVALUATION PROCESS 


CRETERIA QUESTION #1 THE PROGRAM SHOULD DISPLAY AN INTEGRITY OR WHOLENESS 


1st Draft Of . Review Revise Draft 5 Final > Collars 
Parameter Statement Faculty Statement 
_ el ; ) Present Position s©.. 


List of © @) Professional Duties 6)... 
ow estio: oo 
© Graduates ws eee 2 And Activities —> * 
: Graduate School tins 5 on 
Leading To Present Fos 
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Qj >O——©> 


©™) List_of Program ©) Review By Faculty 3 ©=> >C) = 
Gs — 
Competencies List 


CRITERIA QUESTION #2 


THE PROGRAM SHOULD BE COMPETENCY-BASED; AND THE PROGRAM SHOULD 


PROVIDE MULTIPLE EXPERIENCES THAT PERMIT ACHIEVEMENT OF EVERY 


COMPETENCY THAT IS REQUIRED 


4 : 
»” List of Program Order - © ==>) >) List of es (°) 
A. ‘ Competencies Competenci Order Order ed Competenciés = 


' 3 


eriences To Rationale For Pro ie Ma tae 
Provide Achievemen Experience Exclusion > ) >( ) 
! 7 


7 | 
. 7 To Final 
Evaluation Method = (s) 2 Report 
For Achievement 


THE PROGRAM SHOULD BE CAPABLE OF A GREAT DEAL OF 


CRITERIA QUESTION # 3 


INDIVIDUALIZATION 


Pd Minors 
/ 
a. |From: 
r C.QH1,C Rank Pro Competencies in 
af der o: ortance 
oe », 
o oo” 
4 7 : ( ) Check Competencies Able >( ) 
of” aoae : 
- = To P 
_B. fe{Qiestionnaire oO Eertoral = =) 


| *\To~Graduates) Check Competencies 
Tee Received During Program 
key 


we. 
~ 


Check Competencies Cd 
Presently Being Utilized 


Course Evaluation 
Methods : . 


Program Evaluation Report 
Methods 


CRITERIA QUESTION #4 THE PROGRAM SHOULD REST ON SOUND STUDENT RECRUITMENT 


AND SELECTION CRITERIA 


Criteria For Student Rationale For 
Selection Criteria 


( ) Initial Position > ) 
. Pat ~\ 
a ( exe or Would Sent > *) 
’ 2: --o" = 
' List of AowsCionneirs Students to I.U. Re 
Graduates <Pe~Graduates) Have or Would Contact ~ 
~s T.U. When Positions 
Available 


~ 
SA. 
~ 


a: Reason For Atten I. 


CRITERIA QUESTION #5 THE PROGRAM SHOULD PROVIDE FOR THEORY-PRACTICE INTEGRATION 


* Development Theoretical 
Activities Base re 
. 4 
a o 


: Fa leisy ‘4 
Involvement 


B. ticum Theoretical 
Activities se * 


2 r) 


5 
‘eo Student 
Involvement 


CRITERIA QUESTION #6 THE PROGRAM SHOULD POSSESS SOPHISTICATION AND RIGOR 


A. 
From: 
C.Q# 1,C 
Development Competencies 
B.1) oo 
Activities Developed 10 


List Competencies 


3) In Activities Competencies 
velope 


CRITERIA QUESTION #7 THE PROGRAM SHOULD BE INQUIRY ORIENTATED 


From: 
C.0.H1,C 


Competencies > ©. 
— Required 
Y P 
Faculty-Student 
( ) Involvement 7 G y 


B. — fuestionnair 
'o Graduates Since Graduation 


CRITERIA QUESTIC! #8 DOES THE PROGRAM REST UPON A QUALITY FACULTY? 
List of Prog 


Questionnaire To 


Program; Chairmen 


Teaching Effectiveness e 
(Present 
Faculty) 


1) 0 OX ) Research Activities > = --53 ral —— ( 
es ) Service Activities aG a 


(Resigned 
Faculty) | 


2) OR ) Research Activities > = >> ; Summary ( }— 
ie ) Service Activities > a 
Teaching Effectiveness 
(Recruited 
Faculty) ae 
» OE Research Activities SQ 
oa ) Service Activities a ( ye 


CRITERIA QUESTION #9 IS THE PROGRAM SUPPORTED BY ADEQUATE FACILITIES, 


EQUIPMENT, AND OTHER RESOURCES 


A ©) Five Year Budget = G) 
Summary ~S 


. 
~ 
. 


. 


B. Se eS) at er a -2O-=3O> 


7 
7 
7 
7 


Non—Educational Facilities 7 
= () pneemeneaT 4 
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APPENDIX D 


CURRENT IST COURSES 


Indiana University 
Schoo of Education 
Bloomington, Indiana 


1974-75 quidelines 


undergraduate and 


graduate protessiona 


tudy £ + 
study prog 


INS 


al systems technoia 


The vanous undergraduate and graduate 
Protassional-study programs in Instructional Systems 
Technology (IST) currently offered at Indiana University 


are described in this guide 


Each program is specifically designed to prepare 
persons qualified (1) to meet the increasing demands for 
IST specialists and or (2) to contribute to systematic 
research and theory development in the field 


Students interested in the program should apply to 
Indiana University the Undergraduate and Graduate 
Divisions of the School of Education (BS .MS.EdS 
and EdD) or the Graduate School iM A and PhD ) 
The bulletins of both schools are available upon request 


Students will be required to meet admission 
requirements Three exceptions in requirements are 
made in the School of Education Graduate Division, for 
students not planning careers in public schools (1) the 
admission requirement of a teachers certificate or a 
minimum of ten hours in education may be waived 


2) the masters deqree requirement of 4 ¢ 


ourse 


of three specitied helds in education 


may f 


{the three specified teld 


Ne course in one 


privilege of ch 
omparabie 
the University 


ther twe 


om other dep 


ANG (3) Up fo Sia NOurs tn specianzed 
media courses May be counted toward the hours of 


work required outside education 


Because of the nature of knowledge in the instructional 
systems technology field postmaster s courses must 
draw heavily on the theoretical and research literature of 
the appropriate supporting disciplines For example, a 
student interested in a specialist or doctoral-level 
Program in instructional development must have an 
adequate background in the behavioral sciences In the 
admission of students. consideration will be given not 
only to GRE aptitude scores. gradepoint averages, and 
references but also to the amount and quality of course 
work in supporting disciplines on the undergraduate and 
graduate levels 


A number of interesting possibilities for professional 
Careers using instructional systems technology in 
Supportive roles are available Undergraduate students 
interested in a minor in Instructional Systems 
Technology should consult with the Director of the 
Orvision or any faculty member in the Division 


Graduate students should first consult with the 
Chairman of the Division and then with faculty members 
in their curnculum emphasis area (See page 2) 


Doctoral students electing to take a minor in IST are 
required to take at least 15 hours and very possibly 


more if the IST minor will be instrumental in getting their 
positions or if the competencies developed in the IST 
minor represent a substantial segment of their work 
Normally the doctoral minor in IST should include at 
least two courses at the 600 level 


Juestons for tne Qualtying Bacar 


hmeds 


Committee yn the 


tabcommuittee He probably will prepare 


ne ut the questions himselt and contact other 


s tor additional questions He wi 


Drubably ask the person who prep. 
Questions 


10 the Duectors Otice 


PaCN Question t 


ne answer and responses are to be 


twar 


Students in the IST program can complement their 
course experiences in the Audio-Visual Center which is 
functionally and operationally coordinated with the 
Division of Instructional Systems Technology Among 
the purposes of the Audio-Visual Center are (1) to 
provide opportunities for advanced study, experience 
and research in the practical and theoretical aspects of 
Production, selection, circulation and ufifization of the 
major types of audio-visual materials and in the 
organization and administration of an instructional 
materials program and (2) to contribute to instructional 
and research activities of all schools and departments at 
Indiana University by providing faculty members and 
students with professional assistance in "he selection, 
Procurement, and use of audio-visual materials and by 
offering assistance and facilities for the production of all 
types of materials Some students prefer in-depth 
experiences in one operational area of the Center. eg 
production, selection, diffusion, or administration, while 
others preter a wider range of on-the-job experiences 
and during their professional-study career will shift from 
one area of work to another Obviously there are 
advantages to both of these strategies 


Bachelor's Degree Programs 


A minor in the field of Instructional Systems Technology 
is available to undergraduate students Students in the 
Schoo! of Education may wish to combine such a minor 
with a major in a selected teaching area, Radio and 
Television. or Schoo! Library Services In these cases. 
the Indiana Division of Teacher Education and 
Certification requires at least 24 semester hours For 
additional information, see the Bulletin of the School of 
Education, Undergraduate Program 


Courses should be selected from the following in terms 
of candidate s career plans or teaching majors 


Education R423 Utilization of Instructional Materials 
Education A533 Organization and Curncular 
Integration of Audio-Visual Media 
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Division of 
Instructional Systems 
Technology 

and 

Audio-Visual 

Center 


Education A543 


Preparation of Inexpensive 
Instructional Materials 
Production Techniques 

Survey of Audio-Visual 
Communications 

Audio-Visual Communications in 
Formal Education 


Education R544 
Education R546 


Education R566 


A minimum of six semester hours in anthropology 
psychology, and.or sociology One course to be selected 
from instructional systems technology, journalism, library 
science, radio, or computer technology 


Master's Degree Programs 3 


The master's degree program is a one-year graduate 
Program requiring completion of a minimum of 36 
semester hours in graduate courses (without a thesis or 
30 semester hours with a thesis) Students who plan to 
continue for an advanced graduate degree should 
consult with appropriate Instructional Systems 
Technology advisor on masters-level courses required 
for admission to specialist or doctoral degree programs 


Courses should be selected in terms of a candidate's 
educational career plans or curriculum emphasis area as 
suggested by the following representative career areas 


1, Teaching Area/IST Combination 

2. Coordinator of Audio-Visual Materials 

3. Instructional Materials Center Coordinator 
4. Production of Instructional-Materials 

5. Instructional Television Specialist 


Specialist Degree Programs 


The specialist degree is a two-year graduate program of 
65 semester hours with the second-year or 600-level 
courses Carrying prerequisites in Instructional Systems 
Technology, as well as in supporting disciplines, that 
must be taken at the undergraduate and/or master's 
levels Specialist degree majors in instructional design 
and development, product evaluation and curncular 
integration, and systems design and management ot 
learning resources programs usually should include a 
data processing course 


Special forms are available in the Director s Otfice for 
listing specific courses All courses should be selected in 
consultation with the student's advisor 


1, Instructional Systems Technology Courses 


24 hours minimum with selection in accordance with a 
major emphasis in 

(a) Instructional Television 

(b) Instructional Design and Development 


(c) Product Evaluation wand ¢ 

(d) Systems Design and Management 
Resources Programs 

(e) Message Design and Production 

(ft) Oittusion and Adoption of Innovaty nstructiona 


Systems Technology 


Since the Eu S iS a Si ar progr wnere 


number of 600 leve! courses 5! 


sded among 


the total of 24 sem r hours 


2. Foundation Courses 


At least one course in four of tne six areas 
(a) Research 

(b) History Phiiosophy 

ic) Psycnology 

(d) Curriculum 
(e) Educat 

{t) Statistics 


ii Measurement 


3. Courses in Fields Closely Related to 
Instructional Systems Technology 


Nine semester nours either inside or outside the School 
of Education closely related to IST The specific courses 
selected would depend on the primary emphasis area 
the student has selected within IST 


4. Courses in Appropriate Fields Outside of 
School of Education 


Nine semester hours in appropriate fields outside of 
School of Education depending on the primary emphasis 
area within the IST program 


5. Electives Approved by the Student's 
Committee 


Electives should bring the total to a minimum of 65 
semester hours of graduate credit 


Doctoral Degree Programs 


The doctoral program requires the completion of a 
minimum of 90 semester hours of graduate credit It 
should be planned on a three-year basis in order to 
select, on the master 5 level, those courses in IST and 
supporting disciplines needed as prerequisites for the 
second year of graduate work and also to plan for 
second year, 600-level courses needed as prerequisites 
for the third year, 700-leval courses 


Doctoral degree students generally select from doctoral 
curriculum emphasis areas (Numbers 2-6. pages 2-3) 
one curriculum emphasis area in which to concentrate 
and one or more supporting closely related emphasis 
areas one ot which may be selected from the four 
areas described on page 3 A student may, however 
with the advice of his counselor or doctoral committee 
develop his program of courses from two or more of the 
curriculum emphasis areas rather than from one. Such a 
decision should reflect his goals, past experiences, and 
job opportunities. Such individualizing or tailoring of a 
doctoral program does not contradict the curriculum 
emphasis area concept since the emphases represent 
clusters of courses related to developing competencies 
in some of the more frequently recurnng professional 
placement areas and definitely are not required tracks 
Students for the doctoral degree also select two 
supporting minors, one of which must be outside 
education and represent a supporting discipline for the 
desired specialization in educational media. A minor in 
General Communications can be used as a minor 
toward the Ed D degree but not toward the PhD 
degree 


In order to receive adequate counseling, doctoral 
Students should obtain an advisor upon arrival on 
campus After notification of admission to the doctoral 


program students should obtain a doctoral committee 
4s 500N as possible Entering students planning to 
follow a doctoral program should at their earliest 
opportunity 


(1) Complete an application for admission to advanced 
graduate study — be sure that the application is 
complete in ail details 


(2) Request trom the Director of the Division of 
Instructional Systems Technology a pro tem advisor 
pending action on application 


(3) Upon notification trom the Graduate Division that 
application has been approved, obtain a Doctoral 
Committee Membership form for signature of individual * 
doctoral committee members The chairman of the 
committee should be consulted in planning the 
composition of the committee 


(4) Plan program tentatively in consultation with 
doctoral committee chairman 


(5) Plan a doctoral committee meeting in consultation 
with doctoral committee chairman for the purpose of 
program approval 


Curriculum Emphasis Areas 


As indicated in the foregoing brief descriptions of the 
specialist and doctoral degrees. there are six curriculum 
emphasis areas from which the candidate, in 
collaboration with his committee, selects one or a 
combination in an effort to develop the best possible 
specialist or doctoral program in terms of his individual 
experiences, professional goals, and the demands of 
future job opportunities 


The descriptions of these areas follow. Numbers 1-6 
may be selected by specialist degree candidates 
Numbers 2-6 may be selected by doctoral degree 
candidates 


The candidate and his committee will be helped in 
sequencing courses by consulting a tentative long-range 
timetabie tor offering IST courses which ts available 
through the Director s Office 


1. Instructional Television (G. Johnson) 


Specialist Degree The program tor the instructional 
television specialist 1s designed to develop 
competencies for (1) producing instructional television 
programs or (2) supervising the production of 
instructional television programs 


The curriculum in instructional television provides the 
student with experiences in preparing programs, both 
closed-circuit and broadcast, for schools, churches. 
government, the military. and industrial organizations 
Graduates will need a variety of production 
competencies not only in television but also in such 
related media as graphics, motion pictures, and still 
photography. Job functions will include TV direction, 
writing for TV, and message design. 


The need for professionals with competencies in ITV 
Production is increasing. Schoo! systems find a growing 
need for programs closely related to local and regional 
instructional programs. Professional schools in medicine, 
dentistry, and education are using the video tape 
recorder to produce ITV programs which can repeat 
lessons for multiple sections of classes. Often these 
lessons include processes and subject matters which 
would be difficult, time-consuming, and expensive to 
repeat in live broadcasts. Government and industry are 
also making use of the TV medium for the same 
reasons 


Graduate assistantships for those selecting the ITV 
emphasis may be within the TV medium or one of the 
related production areas Involvement in client 


relationships and the many problems in the diffusion and 
adoption of the ITV message would be valuable 
experiences A number of different assistantship 
possibilities would relate to the competencies needed in 
ITV production 


2. Instructional Design and Development 
(Schwen, Davies) 
Specialist or Doctoral Degree Instructional development 
is essentially a systematic decision-making process in 
which the developer takes into account all the classes of 
variables which might affect the quality of the instruction 
under development The developer synthesizes trom 
among these variables those which offer the greatest 
promise for a given learner and learning task. He then 
designs sub-systems for evaluation, logistics and 
management for the instruction He subjects the 
instructional segment to developmental and field testing. 
revising it as required 


The instructional development process has wide 
applicability in colleges, schools, industrial and military 
training, and other instructional settings. 


The degree programs consist of coursework, practica, 
seminars, an internship, and — in the case of the 
doctorate — formal research. Several graduate 
assistantships, which provide opportunity to assist 
faculty in the design of instruction, are available 


3. Product Evaluation and Curricular 
Integration (Guss, Cuttill, Heinich, Johnson, 
Wetmore) 

Specialist or Doctoral Degree. Educational 

product evaluation and curricular integration concerns 

the prediction of learning responses to educational 

media It is distinguished from other curricular emphasis 
areas by !ts special focus on the documentation of the 
educational utility of media and the analysis of the 
intents, outcomes of use, and value judgments 
concerned with educational products (hardware, 
software, and methods) in order to provide a valid basis 
for educational decision-making and for curriculum 
planning and evaluation. Curriculum in this context is 
construed as referring to the sequenced learning and 
informative experiences of individuals and/or groups in 
formal and informal situations 


Training and professional careers in this area can be 
divided into two specializations: (1) the educational 
product evaluator (applied scientist) and (2) the 
researcher in educational product evaluation (basic 
behavioral scientist). Interested students may be able to 
work out programs which result in proficiency in both 
areas 


4. Systems Design and Management of 
Learning Resources Programs (Moldstad, 
Richardson) 


Specialist or Doctoral Degree. Increasing reliance 

upon instructional systems technology in instructional 
development coupled with greater public demand tor 
cost-benefit analysis of its effectiveness has highlighted 
the need for skilled management of integrated learning 
resources programs. The systems design and 
management curriculum is designed to prepare 
personnel (1) to manage or teach others to manage 
basic functions of an integrated learning resources 
Program (staff allocation and utilization, selection and 
distribution of media resources, local production and 
adoption of new media approaches, facilities design for 
use of instructional technology, evaluation, etc.) and (2) 
to acquire competencies related to staff roles of the 
instructional systems administrator as an essential 
specialist on course development teams concerned with 
instructional design, development, diffusion, and 
adoption 


Within the curnculum. competencies would be provided 
for careers varying from (1) managers concentrating on 
applying general pnncipies of management-planning. 
organizing, motivating, and controlling to on-going 
Programs in public schools and colleges and universities 
industrial training, government, religious organizations, 
etc . to (2) systems analysis and research-oriented 
specialists skilled in the development and 
insttutionalization of education systems 


5. Message Design and Production (Fleming, 
Carter, Flaten, Frye, Levie, Niekamp, Pett, 
Stevens, Vuke) 


Specialist or Doctoral Degree With a widesprea 
emphasis today upon the technology of instruction and a 
systematic approach to the development of instructional 
materials, there is an increasing need for message 
designers and production specialists. Graduates of both 
the specialist and doctoral degree programs need to 
have competencies in applying behavioral science 
Principles to the design of instructional messages and in 
Producing various types of instructional materials. The 
doctoral graduate, additionally, needs a broad 
khowiedge of research methodology and its application 
to the study of message design variables 


Both programs offer students opportunities to develop 
competencies in scnpt writing and in the production of 
instructional materials using graphics, still photography, 
Motion pictures, programed instruction, and radio and 
television, Students for the specialist degree must take 
more than one course in two or more of the above 


The doctcral degree program requires, additionally, that 
Students be prepared to conduct research dealing with 
theoretical and professional issues of concern to 
message design and production. The doctoral student 
may choose to concentrate on r,.ethodology in 
experimental, evaluative, analytical, or descriptive 
research 


6. Diffusion and Adoption of Innovations in 
Instructional Systems Technology 
(Jwaideh, Bhola, Knowlton) 


Speciast or Doctoral Degree. Researchers and others, 
especially in recent years, have created much new 
knowledge of potential value to education. A great deal 
of this knowledge has been put into consumable form 
But a persistent problem remains: how to get these 
innovative developments put into practice 


Needed to help solve this problem are educators who 
will take a major responsibility for facilitating the 
adoption of innovative practices — practices that include 
all sorts of tested instructional strategies, but especially 
those that incorporate the use of media, “systems 
packages.” or other technological devices. 


The instructional program for the diftusion-adoption 
(D/A) student has three facets: (1) a series of courses 
designed to develop an understanding of the 
diffusion-adoption process, (2) preparation in connection 
with the technological and other innovations with which 
he will be dealing, and (3) a practicum-seminar- 
internship complex of activities 


Research and Theory Concentration 


A research and theory concentration may be chosen by 
doctoral students in any of the five curriculum emphasis 
areas. It can be integrated with any of these programs. 
It 1s designed to provide competencies in basic research 
methodology and theory. It focuses on research and 
theory in the behavioral sciences, advanced statistics, 
correlation and measurement techniques, and research 
design. In addition students will be expected to acquire 
some facility in computer utilization. They may also 


develop advanced skills in more applied research areas 
of curricular analysis or systems analysis. Students 
should have a substantial foundation in theoretical 
content which has developed in disciplines such as 
psychology, sociology, and linguistics In most cases 
they will have a minor in one of these areas, or in other 
appropriate areas These, together with courses 
available in the student's curriculum emphasis area, are 
expected to prepare the graduate for the task of making 
theoretical and research contributions to education, 
particularly those aspects of education that are related 
to instructional systems technology 


Professional careers with heavy emphasis on 
instructional systems technology research are most 
likely to be found in large universities having doctoral 
programs in the IST field, government-supported 
research and development programs, industry, the 
military services, and schools of various kinds. In such 
Positions, professionals would be responsible for 
designing and conducting either basic or applied 
research, contnbuting knowledge to the field through 
scnolarly writing, and consulting and advising others 
regarding methodology and theory 


Additional Areas for 
Developing Competencies 


One or more courses in IST, as well as additional 
courses in other schools and departments of the 
University, may be used to plan a limited program of 
courses in several competency areas as follows 


1. Computer Technology (Bern) 


The computer is already so demonstrably pertinent to 
the IST field that doctoral and specialist candidates 
would be well advised to take one or more courses in 
computer technology. Prognoses for the future attest to 
the centrality of the computer for the following aspects 
of the IST field: research, administration, information 
Storage and retrieval, simulation, and instruction. The 
computer may become the most versatile and 
indispensable “medium’ yet devised by modem 
technology. Computer technology will be of potential use 
to all IST specialists and a very important area of 
“specialization for some 


2. Educational Media in Areas of Rapid 
Change (Stevens, Pett) 


Work leading toward competency in this area considers 
instructional problems in the light of social systems 
Structure, function, maintenance, and renewal. The 
Processes of enculturation, acculturation, continuity, and 
change are studied to undergird the development of 
skills required for the design of instructional programs for 
areas of rapid economic, social, and political change in 
the United States and abroad 


3. Programed Instruction (Hughes, Brown) 


Programed instruction integrates instructional 

method, SR learning theory, and special matenais 

and devices in a way that has demonstrated success * 
in education, industry, and government. Importantty, 

it has focused the attention of educators on the 

value of specifying objectives in behavioral terms, on 
the merit of testing students to ascertain what 
objectives have been achieved, and on the 

desirability of modifying instruction so that more 
students meet the criterion objectives. 


4. Writing for Educational Media (Carter, 
Fleming, Niekamp) 


The design, production and evaluation of educational 
materials usually require word skills. Such skills are 
developed by applying writing and communications 
Pnncipies in situatons that expioit the unique 


characteristics of chosen media Message objectives 
and media potentialities suggest wnting styles. Although 
most of the experiences offered involve design of film 
Productions, the skills acquired are transferable to other 
media, as well as to genoral academic performance 


Course Descriptions 


R423 Utilization of Instructional Materials 
(3 cr.) 


For preservice teachers. Laboratory experiences in the 
Selection, preparation, utilization and evaluation of 
instructional materials culminate in a simulated 
classroom presentation by each student 


R505 Workshop in Instructional Systems 
Technology (cr. arr.) 


Demonstrations, lectures. and group discussions 
concerning utilization, preparation, and administration of 
audio-visual materials. Guest lecturers and consultants. 
One credit hour for each week of full-time work. 


R523 Utilization of Audio-Visual Materials 
(3 cr.) 


Deals with the planning and use of resources for 
developing instruction in the classroom or other 
locations where leaming takes place. Designed to 
provide the student with experiences which will enable 
him to select, arrange, and utilize a variety of resources 
in a systematic approach to instruction. 


R533 Organization and Curricular Integration 
of Audio-Visual Media (3 cr.) 


P: R523. Organizing, facilitating, and integrating of 
audio-visual materials in school curricula and educational 
programs, defining of purposes of inztruction, planning of 
instructional units, and selecting and evaluating of 
matenals. 


R535 Programed Instruction (3 cr.) 


Students develop original programed materials as they 
learn about the process of instructional programing. 
Their work is in the five stages: task analysis, design, 
6diting, developmental testing, and validation testing. 
Although the relevance of principles of learning is 
considered, the major emphasis is on the mastery of the 
skills and techniques involved in writing programed 
material. 


R537 Writing for Educational Media (3 cr.) 


Principles of script writing for visual and verbal continuity 
and sequencing of ideas. Considers also audience 
involvement factors and organizational pattems as they 
atfect script writing. 


R543 Preparation of Inexpensive 
Instructional Materials (3 cr.) 


Laboratory practice in creating visual instructional 
media, applying lettering, coloring, and mounting 
techniques. End products include pictures, maps, charts, 
and graphs for projected and nonprojected use. Includes 
application of a basic model of graphic communication. 


R544 Production Techniques (3 cr.) 


Basic principles, skills, and techniques of photography 
and their application to problems of communication and 
instructional development. 


R546 Survey of Audio-Visual 
Communications (Theory) (3 cr.) 


Considers concepts, principles, and theoretical 
orientations drawn from the fields of psychology, social 
Psychology, anthropology, communications and general 
systems. On completing the course, students should be 
able to identity the authors of, describe, compare, and 


place in nistoncal pe 
above fields as they relate to the rc Wovanous 
of communication 


spective Contabatons made by the 
nadia 
and technologies 
learning and culture transmission 


the processes of 


R550 Introduction to Diffusion and Adoption 
of Educational Innovation (3 cr.) 


P  Graduat nding Discusses the 
context of diffusion and adoption of innovation in 
education Presents an overview of education s 
expenence with diffusion and adoption introducing briefly 
the technological sociological psychological and 
political aspects of the process of diffusion and adoption 


of education and innovation 


R555 Radio and Television in Education 
(3 cr.) 


Communication in the learning process, radio and 
television transmission, planning. scheduling, classroom 
utilization, and evaluation of educational programs, case 
studies and research findings Not open to those who 
have taken Education R453 


R558 Utilization of Instructional Television 
(3 cr.) 


The contributions of instructional television and its 
integration into formal and informal learning situations 
characteristics of telecasts. including those intended for 
supplementation or enrichment, major, resource. and 
direct teaching, and observation, utilization, and 
evaluation of instructional telecasts 


R566 Audio-Visual Communications in 
Formal Education (3 cr.) 


P R523 Applies basic concepts of communication to 
problems in teaching and learning with school and adult 
audiences Discusses the role and contribution of 
instructional technology in problem solving, attitude 
formation. and motor learning Considers vanous 
systems approach models to instructional development 
and offers students a simulation exercise in applying a 
systems model in planning, producing. and testing a unit 
of instruction Each student develops an 8-10 lesson 
unit utihzing a model of his choice 


R572 Principles of Photographic Production 
(3 cr.) 


P R543 and A544 Principles of black and white and 
color photography as applied to the design and 
development of instructional materials Laboratory 
experience includes work with display photographs. slide 
sets, filmstnps, and transparencies 


R573 ‘Principles of Graphic Communications 
(3 cr.) 


P_ A543 and A544 Language and design of graphic 
communication, principles from psychology and the 
visual arts, basic design pninciples, exploration through 
laboratory exercises in the creatrve development of 
Opaque and transparent visual imagery 


R574 Educational Motion Picture Production 
(3 cr.) 


P_ R544 of equivalent An introductory laboratory-type 
course designed to develop individual skilis in 
communicating with moving images Practical application 
of motion picture technology in the production of 
Systematically designed messages Each student 
Produces a short him, considers appropriate research 
findings, identifies relevant evaluative cntena, and 
formulates proposals for revision 


R590 Research in Instructional Systems 
Technology (cr. arr.) 


P Consent of instructor 
Individual research 


R595 The Learner, Media and Research 
(3 cr.) 


Studies the research history of instructiond! systems 

technology Introduces the language and approach of 

science and applies this understanding to the analysis of 

problems in instructional systems technology Introduces 
urrent relevant research in the disciplines foundational 

to instructiondl systems technology This course has a 

aburatory component 


R597 Practicum in Medical Technology 
Teaching Skills (1-4 cr.) 


R599 Master's Thesis in Instructional 
Systems Technology (3 cr.) 


Individual study of a specific problem May be an 
organized scientific contnbution or a comprehensive 
analysis of theory and practice in a specific area 


R610 Learning Environments Design (3 cr.) 


P Approved graduate courses in each of the following 
— media production and utilization, educational 
Psychology. educational measurement, statistics, plus 
consent of instructor Pnnciples and practice of 
environmental design Study of interrelationships among 
environmental vanables Use of decision models in the 
design process Design, construction, and testing of 
learning environments representing alternative profiles of 
variables 


R620 


P R610 Principles and practice of analysis of 
instructional tasks Study of task configurations, 
taxonomies for task classification, and task sequencing 
Extensive analysis of actual instructional behaviors 


Instructional Task Analysis (3 cr.) 


R630 Learner Analysis in the Instructional 
Technology Process (3 cr.) 


P R610 Methods for utilizing aptitude data in the 
instructional technology process. Use of 
criterion-referenced instruments to diagnose entry 
behavior and place students in the instructional 
environment Use of task and aptitude information to 
create instructional strategies differentially effective for a 
wide range of students 


R634 Theory and Principles of Message 
Design (3 cr.) 


P. A behavioral science course and a production 
course. Relates selected research-based principles from 
behavioral science to the problems and processes of 
designing instructional messages. Units in perception, 
learning, persuasion, and structure of knowledge, 
laboratory projects in designing messages in students 
subject area 


R636 Technology of Education (3 cr.) 


P Three IST courses. The course explores applications 
of principles of technology to education and examines 
changes likely to occur in education as an emerging 
technological society The relationship between science 
and technology with specific reference to education and 
the technological ongins of “systems” approaches are 
examined 


R640 Educational Media in Areas of Rapid 
Change (3 cr.) 


P R546. Considers factors of relevance to the uses of 
media and to the design of instructional mediation 
systems in changing societies. 


R646 Research in Verbal and Visual-Iconic 
Signs in Instructional Message Design (3 cr.) 


PA general. introductory psychology course. or one of 
A535, R546, A595, R634 Examines the relative 


. 

strengths and weaknesses of visual-iconic (eg 
pictorial) and digital (primarily verbal) signs, for varying 
feaching-learning purposes, especially as these are 
contingent on the learners developmental level The 
course 1s designed with both the instructional message 
designer and the Mass Communications major in mind 
Introduces certain methods relevant for research in 
nonverbal Communications 


R650 Diffusion and Adoption of Educational 
Innovations — Models, Techniques and 
Applications (3 cr.) 


P R550 Discusses some important change models tor 
understanding and planning diffusion and adoption of 
educational innovation. Uses of these models and 
various diffusion and adoption techniques in specific 
educational situations are demonstrated 


R655 Educational Writing for Radio and 
Television (3 cr.) 


Planning and writing the lecture-demonstration, the 
semi-scripted program, and the tully-scripted 
documentary program, styles of writing as related to 
@ducationai objectives, program design in relation to 
Production procedures 


R660 The Individual, the Group and Planned 
Change: Social-Psychological Determinants (3 
cr.) 


P R550. Considers psychological principles 
(perception, motivation, learning) and 
social-psychological forces (interpersonal and small 
group dynamics), relevant to understanding resistance 
to change. Integrates these pnnciples in ways to 
facilitate an understanding of innovation diffusion. Points 
up educationally important questions in the 

diffusion adoption area that need researching 


R665 Administration of Audio-Visual 
Communications Programs (3 cr.) 


P- Basic courses in instructional systems technology 
Applies basic principles of management in the design 
and operation of varying types of media programs in 
various institutional settings. Identifies primary functions 
performed by instructional media personnel and 
considers various organizational patterns, administrative 
Strategies, and alternative structures for achieving and 
evaluating each function. Each student develops a 
detailed 3-5 year proposal for a learning resource 
Program 


R670 Sociological Aspects of Innovation 
Diffusion in Education (3 cr.) 


P: R550 Sociological characteristics of school systems 
which are relevant to educational innovation, including 
new technology Educational institutions as social 
systems (inter-role relationships, communications 
Networks, opinion leadership, social controls) 
Institutional barriers to change, including political and 
economic factors. Role of foundations, accreditation 
agencies, professional organizations, governmental 
agencies, etc. Applications from organizational behavior. 


R671 Advanced Production | — Motion 
Picture Emphasis (3-5 cr.) 


P A574 and/or consent of instructor Message design 
and development pnnciples applied to the creation of 
instructional films. Students work in teams on the 
production of professional quality sound motion pictures 


R671 Advanced Production | — General 
Instructional Materials Emphasis (3 cr.) 
P. R572, R573 or A574 andor consent of instructor 


Message design. audio, graphic, photographic, and 
motion picture principles and techniques applied to 


pment ' 


Nateanials to meet specitic objectives 


R672 Advanced Production ll — Motion 
Picture Emphasis (3-5 cr.) 


P R671 Message design and development principles 


the cr yt instructional films St 


sound motion pictures 


R672 Advanced Production Il — General 
Instructional Materials Emphasis (3 cr.) 


P R671 A practicum course in which students work 
wth teacher or other clients to uce instructional 
matenals to meet specific obje es Seminar sessions 


focus on theoretical and applied aspects of production 


R674 Research Methods in Instructional 
Systems Technology (3 cr.) 


P P50! _— Initially, the nature of science is examined as 
well as the significance of science for education Then 
topics of research design are taken up, with special 
emphasis on experimentation Finally students 
participate in the various aspects of a research project 
including the writing of a report on the research done 


R773 Seminar in Instructional Systems 
Technology | (2 cr.) 


A research seminar required of all doctoral candidates 
designed to develop the ability to examine cniically 
Student and faculty research in instructional systems 
technology in progress 


R774 Seminar in Instructional Systems 
Technology I! (2 cr.) 


P= Major portion of doctoral major and minor courses 
completed and consent of chairman of doctoral 
committee Deals with researchable topics in 
instructional systems technology Designed to help 
doctoral students identity researchable topics, develop 
research designs, and prepare research proposals 


R799 Doctor's Thesis in Instructional 
Systems Technology (cr. arr.) 


Credit may be earned over a period of several 
semesters. The thesis may be an organized scientific 
contribution or a comprehensive analysis of theory and 
Practice in a specific area 


Academic Counselors 


Otfices of most of these academic counselors are in the 
Student Services Building located at Kirkwood and 
Indiana Avenues and are most easily located by entering 
the Indiana Avenue entrance Their telephone numbers 
can be found in the Indiana University Telephone 
Directory or by calling the University operator. It is 
requested that students wishing conferences with 
academic counselors make appointments by telephone 


Bachelor's Degree Options Counselors 


IST Minor Heinich 
Molenda 
Pett 
Sherman 
IST Radio and Television Sherman 
Johnson, George 
IST School Library Services Sherman 
Guss 


Master's Degree Options Counselors 


Teaching Area IST Combinations Guss 
Flaten (Botany, Zool.) 
Vuke (Bio. Sct.) 
Coordinator of Audio-Visual Materials Guss 
Moidstad 
Instructional Matenals Center Coordinator Guss 
Moldstad 
Production of Instructional Materials Carter (Mot. Pic.) 
Flaten (Still Photo) 
Frye (Graphics) 
Levie (Mot. Pic.) 
Pett (General Prod.) 
Stevens (Still Photo) 
Vuke (Mot. Pic.) 
Instructional Television Specialist Johnson, George 
Master's International Students Pett, Stevens 


Ed.S., Ed.D., Ph.D. Degree Emphasis Areas Counselors 


Instructional Television Johnson, George 

Instructional Design and Development Davies 
Schwen 

Product Evaluation and Curncular Integration Guss 
Heinich 

Systems Design and Management of Learning Heinich 

Resources Programs Moldstad 
Richardson 

Message Design and Production Carter (Mot. Pic.) 
Flaten (Photo) 
Fleming (Mes. Des.) 
Frye (Graphics) 
Levie (Mot. Pic.) 
Pett (Mes. Des.) 
Stevens (Mes. Des.) 
Vuke (Mot. Pic.) 

Diffusion and Adoption of Innovations Bhola 

in Instructional Systems Technology Jwaideh 

Knowlton 

IST in Developing Countries Pett 
Stevens 


Faculty 


Bern, Henry A. Ph OD. Professor of Education 

Bhola. Harbans S , PhD. Associate Professor of 
Education 

Brown, Jerry, Ph D. Assistant Professor of Education 

Carter, Ledtord C , MS in Ed. Assistant Professor of 
Education 

Cutt, Willam J. MS in Ed, Lecturer in Education 

Davies. Ivor K . Ph D. Professor of Education 

Fans, Kenneth G , Ed D. Dean for Learning Resources 
Office of the Vice-President and Dean for Academic 
Atfairs, and Professor of Education 

Flaten, Ciarence M , PhD, Professor of Education 

Fleming, Malcolm L . EdD. Professor of Education 

Frye, Harvey RMS in Ed, Associate Professor of 
Education 

Guss, Carolyn. Ed D. Professor of Education 

Heinich, Robert, Ph D. Professor of Education 

Hughes. Lawson H_ PhD. Associate Professor of 
Education 

Johnson, George C .AM., Professor of Education 

Johnson, Stephen C Ed 0. Assistant Professor of 
Education 

Jwaideh, Alice R.Ph D Associate Professor of 
Education 

Knowlton, James Q , PhD. Professor of Education 

Larson, Lawrence C AM. Professor Ementus of 
Education 

Levie. W Howard, PhD. Assistant Professor of 
Educaton 

Moldstad, John A., Ed D, Professor of Education 

Molenda, Michael, Ph 0, Assistant Professor of 
Education 

Niekamp. Walter E EdD. Lecturer in Education 

Pett, Dennis W. EdD. Associate Professor of 
Educaton 

Richardson, Edgar L ,Ed OD, Associate Professor of 
Education 

Schwen, Thomas M, EdD . Assistant Professor of 
Education 

Sherman, Mendel, Ed 0. Director Division of 
Instructional Systems Technology, and Professor of 
Education 

Stevens, Warren D . Ed D, Professor of Education 

Stockton, Philip W.AM. Lecturer in Education 

Thiagarajan, Sivasailam, Ph D , Assistant Professor of 
Education 

Vuke, George J. EdD. Associate Professor of 
Education 

Wagner, Chris M. AM. Lecturer in Education 

Wetmore. Patncia C Ed D. Assistant Professor of 
Education 


General Requirements 
and Information 


Requirements for advanced graduate degrees with a 
major in Instructional Systems Technology, descnptions 
of all courses in education, and other information 
concerning enroliment in the Graduate Division of the 
School of Education or in the Graduate School are given 
in the Graduate Division Bulletin of the Schoo! of 
Education, which can be obtained by writing directly to 
the School of Education 


Although the Aptitude Test of the Graduate Record 
Examination is not required for admission to the Master 
of Science in Education degree program, a student's 
chances of obtaining favorable consideration for a 
graduate appointment at this level are greatly enhanced 
it his test scores accompany his application. The GRE 
Aptitude Test is required for admission to the specialist 
and doctoral programs Normally a minimum total 
standard score of 1,000 on the Verbal-Quantitatve 
Aptitude Test of the GRE is required tor admission to a 
doctoral program. For application forms and information 
about the test and the dates it will be administered, 
please write to the Educational Testing Service. Box 
955, Princeton, New Jersey 08540, or 1947 Center 
Street, Berkeley, California 94704 


Individuals interested in further specific information 

concerning the undergraduate and graduate programs in 
IST at Indiana University should include in their inquines 
information concerning previous training and experience 


Requests for applicaton blanks and further information 
concerning graduate assistantships, fellowships, and 
part-time positions should be sent to 


Dr. Mendel Sherman, Director 
Division of Instructional 
Systems Technology 

School of Education 

Indiana University 
Bloomington, Indiana 47401 


